Repo Cache¶
Workers don't re-clone a repository for every scan. All workers share one cache directory (a local path, or a mounted ReadWriteMany PVC in Kubernetes) plus Postgres-based locks. This lets a fleet of workers on different pods collaborate on one set of clones and coverage profiles.
On-disk layout¶
<cache.directory>/
<repoID>/ # shallow clone, one per repo (repoID = sha of the URL)
coverage/
<repoID>/
<sha>.out # go test -coverprofile output, keyed by commit SHA
- Clone —
<cache.directory>/<repoID>, one per repo. - Coverage —
<cache.directory>/coverage/<repoID>/<sha>.out. Coverage profiles are stored per commit SHA, so a commit that was already tested is never re-tested. Older profiles are pruned tocache.coverage_keep(default 10) most recent.
Flow per scan¶
For each job a worker handles:
- Acquire the repo lock (Postgres, see Locking).
- Prepare the clone — clone if missing; otherwise fetch + hard-reset to the tracked ref (the task
ref, the per-repo--refoverride, or the clone's default branch). - Coverage — if a profile exists for the commit SHA, reuse it and skip
go testentirely; otherwise rungo test -coverprofileonce and store the result. - Scan — run
go-crap scan --coverage-profile <profile>(always), so scores always reflect the tests. - Save — upsert the analysis into Postgres under its deterministic ID.
- Release the lock.
If the ref or commit changes, the clone is updated in place; if the update fails, the worker removes the directory and re-clones fresh.
Locking¶
Locks live in Postgres (repo_locks), not on the filesystem, so any worker on any pod can take them — a ReadWriteMany volume would not give cross-pod filesystem locking.
- Each lock is a lease (default
cache.lock_lease30s) that the holder refreshes on a background loop (~10s cadence) while it works. - If a holder's refresh stops (crash, eviction), the lease expires and the lock becomes stealable.
- A worker that can't acquire the lock waits up to
cache.lock_wait(default 10m) before failing the task. - In-flight work aborts immediately if the lock is lost mid-scan.
repo unlock <url> --yesandrepo delete <url> --yesforce-clear a stuck lock (see repo command).
Per-repo overrides¶
Stored in Postgres (repo_configs) and applied to every scan of that repository. Set them with the CLI or the REST API:
go-crap-tracker repo config set https://github.com/org/repo \
--ref release \
--test-args "-race ./pkg/..." \
--crap-args "--exclude,legacy/.*" \
--submodules \
--timeout 30m \
--git-timeout 30m \
--go-flags "-p=1 -parallel=2"
| Override | Effect |
|---|---|
ref |
branch/tag to track instead of the task's ref |
test-args |
extra args after go test |
crap-args |
extra go-crap flags (comma-separated) |
submodules |
clone/update with git submodules |
timeout |
per-repo go test timeout ("0" = global default) |
git-timeout |
per-repo git operation timeout ("0" = global default) |
go-flags |
GOFLAGS for the repo's go test / go-crap subprocesses; empty = Go default parallelism. See Constrained Environments |
repo config set is a partial upsert — only the flags you pass change; the others keep their stored value. Remove all overrides with repo config rm <url>.
Tuning knobs¶
| Config | Default | Meaning |
|---|---|---|
cache.directory |
(empty → $TMPDIR/go-crap-tracker-cache) |
shared cache root |
cache.lock_lease |
5m |
per-repo lock lease, refreshed while work runs |
cache.lock_wait |
10m |
how long a worker waits for a busy repo |
cache.test_timeout |
10m |
timeout for go test -coverprofile |
cache.git_timeout |
5m |
timeout for git operations (clone/update/reset) |
cache.coverage_keep |
10 |
coverage profiles kept per repo (per commit SHA) |
→ Configuration for the full list and env-var equivalents.