Architecture¶

go-crap-tracker is a two-process system coordinated through NATS JetStream and Postgres. The control plane (serve) accepts work and tracks state; workers (worker) do the scanning. They never talk to each other directly — everything flows through the message bus and the database.

Components¶

Component	Role
`serve` (control plane)	HTTP API (REST + SSE + web UI), the reconciler, the worker registry, and the migration runner
`worker`	NATS JetStream consumer that claims tasks, clones, tests, scans, and saves results
NATS JetStream	Durable job queue + event bus (two streams)
Postgres	Durable store for tasks, results, locks, per-repo config, and repos

┌─────────────────┐     ┌─────────────────────┐     ┌──────────────────┐
│     serve       │────▶│  NATS JetStream     │────▶│     worker(s)    │
│  (control plane)│◀────│  (jobs + events)    │◀────│  (scan engine)   │
│  API / UI / SSE │     └─────────────────────┘     └──────────────────┘
│  reconciler     │                                       │
│  migrations     │                                       │
└─────────────────┘                                       │
        │                                                 │
        ▼                                                 ▼
┌──────────────────────────────────────────────────────────────────┐
│                        Postgres                                  │
│  scan_tasks │ repo_analyses │ repo_configs │ repo_locks │ repos  │
└──────────────────────────────────────────────────────────────────┘

NATS JetStream¶

Two streams, created on demand by whichever process starts first.

Job stream — `crap_jobs_scan`¶

Subject	Purpose
`crap.jobs` (and `crap.jobs.>`)	scheduled scan jobs

A pull consumer crap-workers is the single shared durable consumer:

Setting	Value
Durable	`crap-workers`
Filter subject	`crap.jobs`
Ack policy	explicit
Max deliver	`3`
Ack wait	`10m`
Deliver policy	new

Workers fetch from the shared consumer (PullMaxMessages(1)) in a reconnect loop, so each job is handed to whichever worker is free and not redelivered while legitimately in flight (10-minute AckWait is well above the 5-minute scan cap).

Event stream — `crap_events`¶

Subject	Purpose
`crap.event` (and `crap.event.>`)	task status updates

The control plane's crap-sse consumer (AckNone) reads the event stream and relays each event onto the SSE hub, so every browser subscriber sees queued → running → completed live.

Worker control subjects¶

Subject	Purpose
`crap.worker.hb`	worker heartbeats (id, host, pid, cpu, status)
`crap.worker.cmd.<workerID>`	per-worker commands (pause / unpause)

Each worker only subscribes to its own crap.worker.cmd.<workerID> subject, so pausing one worker leaves the rest consuming.

Postgres¶

Table	Contents
`scan_tasks`	task state machine (status, attempt, worker, timestamps, error)
`repo_analyses`	one row per analysed commit — scores, metadata, full `report-v1` JSON
`repo_configs`	per-repo overrides (ref, test args, crap args, submodules, timeouts, go flags)
`repo_locks`	distributed, lease-based repo locks (owner, host, expiry)
`repos`	canonical repo registry (id → url, display metadata)

Analyses are upserted by a deterministic ID (sha256(repo_url + commit_sha)[:16]), so re-scanning the same commit replaces the record instead of duplicating it.

A scan, end to end¶

sequenceDiagram
    participant C as Client / UI
    participant S as serve
    participant N as NATS JetStream
    participant W as worker
    participant P as Postgres

    C->>S: POST /api/tasks
    S->>P: insert scan_task (pending)
    S->>N: publish crap.jobs
    S-->>C: 201 {id, status: queued}

    N-->>W: worker pulls job
    W->>P: claim_if_queued (queued -> running)
    W->>N: publish crap.event (running)
    W->>P: acquire repo lock
    W->>W: clone/fetch + go test + go-crap scan
    W->>P: upsert repo_analyses
    W->>P: complete (running -> completed)
    W->>N: publish crap.event (completed)
    W->>P: release lock

    N-->>S: crap-sse consumer reads event
    S-->>C: SSE push (live status)

Design notes¶

Swappable infra. Core logic in internal/core depends only on port interfaces (internal/core/ports); the NATS, Postgres, cache, and runner adapters implement them. That keeps infrastructure replaceable without touching the domain.
Shared cache. Workers share one clone/coverage cache directory (typically a mounted PVC) plus Postgres locks, so repeat scans of the same commit skip both the clone and the go test run. See Repo Cache.
Bounded work. Job retries are capped at the task level (max_attempts 3) and the transport level (MaxDeliver 3); the reconciler expires tasks stuck in running. See Task Lifecycle.