Metrics

Measuring an infrastructure factory

A factory that cannot state its own lead time is a ticket desk with better tooling. Six metrics, each with a counterpart on the software side, and the arithmetic that shows why lead time is the one that matters.

Metrics

Measure it like a factory

The software-factory crowd brings numbers: pull requests per week, tokens per engineer, 3 to 5x2. An Infrastructure Factory needs its own set, the equivalent of DORA for environments. Six are enough to start, and each has a counterpart on the software side.

Counterpart: lead time for changesLead time to a compliant environmentFrom intent submitted to environment handed over, with its evidence attached.Factory target: hours, measured per contract. Days means a ticket is still in the path.
Counterpart: deployment frequencyEnvironments per day per platform engineerDelivered environments, normalized by the people who run the factory.Why it matters: it is the number that justifies platform investment as agent demand scales.
Counterpart: PRs merged with no human reviewZero-touch rateShare of requests fulfilled with no human action anywhere in the path.Why it matters: its inverse is the ticket count. An agent at 3am only ever gets the zero-touch share.
Counterpart: change failure ratePolicy-violation rate on agent requestsShare of agent-originated requests the pre-check refuses, broken down by reason.Read it carefully: a rising rate usually means the contract is unclear to the agent, and rarely that the agent is reckless.
Counterpart: review pickup timeTime to ownerTime until a cost center and a compliance owner are attached to the environment.Factory target: zero. Ownership is a default of the golden path, never a step after it.
Counterpart: age of unmerged workOrphan rateResources alive past their TTL with no reachable owner.Why it matters: it is the day-2 health of the factory, and the first number finance will ask for.

Little's law for environment requests

120requests open at any moment with a ticket in the path
120In flight · ticket
0.6In flight · factory

At 40 requests a day and a lead time of 3 days, 120 environment requests are open at any moment. Through a factory with a lead time of 20 min, fewer than one is.

Work in progress = arrival rate × lead time. Little's law holds for any stable queue, whatever is in it. The inputs are yours; the arithmetic is the only claim.

What to measure

Measure outcomes, not tokens

Counts that are easy to collect are easy to game. Three measures survive that test.

Human interventions per environment

How many times a person had to act between intent and hand-over. The counterpart of Lloyd's human interactions per pull request in the software factory.36 Zero is the target; the trend is the metric.

Lead time

From intent to a compliant environment, measured per contract. It is the one number that tells a software factory whether the floor keeps up.

Zero-touch rate

The share of requests fulfilled with no human in the path. An agent at 3am only ever gets this share.

Environments created, tokens spent and pull requests raised are activity, not outcomes. Palmie makes the Goodhart point for the software factory: once a count becomes the target, the factory optimises the count.41 Count environments and you get environments. Measure lead time and interventions, and you get a factory.