Baseline of the build and the checks

scripts/baseline.mjs times the steps that an agent runs on a repository: the clone, the checkout, the dependency install and the checks. It gives the numbers that a change of the build setup is compared with.

It runs on a fresh clone from git. It uses no cluster and no CI.

node scripts/baseline.mjs --spec <spec.json> [--out <results.json>]

The command prints one markdown table for each repository. --out also writes the samples as JSON.

The cache states

Each measurement runs in three states:

State	Cache	Checkout
cold	Empty	New
warm	The cache of the cold run	New, at the same path
cached	The cache of the warm run	The same checkout, with no change

A new checkout has new file times and no installed packages. The warm state shows what an agent gets when it starts a task in a new worktree. The cached state shows what a tool does when nothing changed.

The script writes its clone, its checkouts and its caches under workDir. It removes them when a repository ends, also when a step fails. It refuses to remove a path outside workDir. The commands of the spec write where they write.

The spec

{
  "workDir": "/var/bench",
  "runs": 3,
  "repos": [
    {
      "name": "example-app",
      "url": "git@example.com:acme/example-app.git",
      "ref": "refs/heads/main",
      "sshCommand": "ssh -i /home/user/.ssh/id_example -o IdentitiesOnly=yes",
      "runs": 1,
      "cache": { "GOCACHE": "go-build", "GOMODCACHE": "go-mod" },
      "steps": [
        { "name": "go mod download", "command": "go mod download" },
        { "name": "npm ci (web)", "command": "npm ci --no-audit --no-fund", "cwd": "web" },
        { "name": "go test", "command": "go test ./..." }
      ]
    }
  ]
}

Field	Meaning
workDir	An absolute path. Put it on a disk with room for the caches.
runs	The number of runs of each state. Default 3. A repository can set its own runs, for a build that takes long.
repos[].url, ref	The remote and the ref. The script pins the commit of the ref at the start and prints it.
repos[].sshCommand	Optional. The value of GIT_SSH_COMMAND for the clone.
repos[].cache	Environment variables that point to a cache folder. Each value is a relative folder inside the cache of the run. A cold run starts with it empty.
repos[].steps	The commands, in order. cwd is a relative folder inside the checkout. env adds variables.

Keep the spec of a real repository outside the desk repository.

The table

Each cell shows the median time of the runs that succeeded. The range, the number of runs and the median 1-minute load of the machine follow. A run that failed does not count in the times. The cell adds **n failed**. A cell with no successful run reads **all n failed**. A command that exited with an error, or is missing, is a failed run.

The footer shows the number of cores and the highest 1-minute load that the script saw. A shared machine changes the times, so compare runs that have a similar load.

The script times git clone of all the history, git worktree add, and each step of the spec.