chore(ci): tear down ki-ci stack after each build #110
Merged
syntropicagent opened 2 weeks ago

CI cleanup — tear down ki-ci stack after each build

Problem: The CI job creates a disposable compose stack (ki-ci: database, clickhouse, cache, directus, gateway, grpc) against the host daemon via the mounted docker socket. The buildspec never tore it down, so 6 containers lingered between builds, consuming ~1-2 GB RAM 24/7. This contributed to the KI-453 deploy-staging timeout: the OneDev agent couldn't allocate a job container while the host was memory-exhausted (7.8 GB total, ~250 MB free, 4 GB swap in use).

Fix: Add a cleanup step after "All CI checks complete":

docker compose -f docker-compose.yml -f docker-compose.ci.yml down -v 2>/dev/null || true

|| true ensures a cleanup failure never fails the build. The .env (with secrets) exists during the build, so compose can interpolate at teardown time.

Verified:

  • tod build check-spec passes
  • Existing 6 leftover ki-ci containers force-removed from cerberus (memory freed)
  • The next CI run will self-clean

Follow-up (not in this PR): the same teardown pattern could be applied to CI build-cache volume hygiene if disk (58/96 GB) becomes a concern.

Commits were merged into target branch
  • chore(ci): tear down ki-ci stack after each build
    The CI job created its compose stack (ki-ci: database, clickhouse, cache,
    directus, gateway, grpc) against the host daemon via the docker socket, but
    never tore it down — 6 containers lingered between builds consuming ~1-2 GB
    RAM 24/7, contributing to host memory pressure (KI-453 deploy-staging
    timed out in 'Pending resource allocation' while memory was exhausted).
    
    Add a cleanup step after 'All CI checks complete' that runs
    'docker compose ... down -v' on the ki-ci project, with || true so a
    cleanup failure never fails the build. Confirmed the spec validates;
    also force-removed the existing leftovers on cerberus.
    2 weeks ago
  • syntropicagent merged 2 weeks ago (merge commit: 02513190 )
1/1
Submitter syntropicagent
Target main
Source chore/ci-cleanup
Jobs
Merge Strategy
Create Merge Commit
Watchers (1)
Reference
pr KI-110
Please wait...
Connection lost or session expired, reload to recover
Page is in error, reload to recover