A developer platform needs an operating review, not a dashboard wall. The point is to decide what to improve: a slow feedback loop, an unreliable paved road, an unclear ownership boundary, or an adoption problem hiding behind a tool request.
That distinction matters because platform measurements are easy to turn into theater. A graph can be accurate and still be useless. Build duration can fall because teams stopped running a meaningful test suite. Portal usage can rise because a workflow became mandatory. A quarterly satisfaction score can improve while the one service that matters most remains miserable to ship.
Treat platform health as a decision system. Pick a small number of signals that describe the developer journey, look at them together, ask what decision each could change, and pair every measurement with a concrete intervention. The goal is not to prove that the platform team is busy. It is to make the reliable path easier and delivery constraints visible.
Start with the developer journey, not the available telemetry
Most platform work touches a repeated path: create a service, make a change, verify it locally, get it reviewed, send it through CI, deploy it, and understand what happened when it fails. A health review should follow that path rather than mirror the organization chart or the vendor console.
Write down the few outcomes the platform is responsible for helping teams achieve:
- A new engineer can make a safe first change without a private scavenger hunt.
- A routine change gets useful local and CI feedback quickly enough to preserve context.
- Teams can use supported deployment and observability paths without negotiating custom exceptions every time.
- A platform failure is visible, owned, and recoverable before every product team invents a workaround.
Those are capabilities, not individual productivity targets. Do not rank engineers by command frequency, pull-request count, hours in an IDE, or who used an internal tool. The platform owns the quality of the path; people own different kinds of work, in different systems, at different points in a project. Conflating those facts produces surveillance with a nicer chart.
The same boundary makes this article different from measuring AI-assisted engineering productivity without turning developers into telemetry. That question asks whether a workflow is helping delivery without dehumanizing the people doing it. This one asks whether the shared system makes safe work practical at all.
Use four signal families
No single metric represents a platform. A compact review can use four signal families, each with a clear decision attached.
| Signal family | Useful question | Example evidence | Decision it can inform |
|---|---|---|---|
| Flow | Where does routine work wait? | Local verification time, CI queue and completion time, deploy lead time | Invest in cache, test selection, capacity, or workflow simplification |
| Reliability | Does the paved path behave predictably? | Flaky-test rate, failed deployment rate, platform incident recurrence | Fix a known failure mode before adding another capability |
| Experience | Can engineers complete the intended workflow unaided? | Task-based feedback, support themes, documented escape hatches | Improve defaults, documentation, or the interface |
| Adoption and coverage | Is the supported path serving the right work? | Eligible-service coverage, successful self-service completion | Prioritize migration support or retire a misleading “standard” |
The examples are deliberately not universal targets. A five-minute build may be fine for a release artifact and intolerable for a tight edit-test loop. A low deployment failure rate may conceal a manual approval process that keeps risky releases from happening at all. Use baselines, segments, and the workflow context before declaring a number good or bad.
Make every signal earn its place
A signal belongs in the review only if it can produce a decision. The fastest test is to finish this sentence: “If this changes materially, we will consider doing ___.” If the blank is empty, the chart is probably decoration.
For a build platform, a useful pair might be median local verification time and the slower tail for a representative test command. The median tells you whether ordinary work is getting smoother; the tail tells you whether some repositories or environments are being stranded. If the tail grows after a toolchain upgrade, investigate the affected cohort rather than celebrating an unchanged median.
For a service template, count successful end-to-end creation attempts and inspect the failures. A completed template generation is not adoption if teams immediately replace the generated CI, deployment, or observability plumbing. Follow the path one step farther: did the service reach the supported deployment route, and could its owner diagnose its first failure?
For a self-service portal, sample real tasks. Ask a few teams to create an environment, request a permitted capability, or find the ownership and escalation path. Time-to-completion and the points where they ask for help often explain more than page views. The platform team should be curious about the workaround; a workaround is a product-research interview, not an act of disloyalty.
Review a slice, then choose one intervention
Do not wait for a perfect data warehouse. Run a monthly or quarterly review around a representative slice of the developer journey, with platform owners and a few users who actually traverse it. Bring a short written packet: the baseline, the change since the last review, known caveats, and the one decision each signal might support.
The review should end with a small number of named bets:
- State the constraint in plain language: “Integration tests spend too long waiting for scarce macOS runners.”
- Name the proposed intervention: add capacity, split a test class, improve test selection, or change the workflow.
- Predict the effect and the guardrail: reduce queue time without reducing test coverage or making failures harder to reproduce.
- Set a review date and identify the owner who will report what happened.
This is a change budget for the platform itself. Giving AI coding agents a change budget makes the same operational point for automation: bounded changes preserve the evidence needed to learn. A platform team that starts six metrics projects and three migrations at once will have a dashboard, but not a causal story.
Segment before you average
Aggregates hide the teams most likely to route around you. Segment by workflow or technical shape when it changes the intervention: service versus mobile client, supported build path versus legacy path, small repository versus monorepo, first deployment versus routine deployment. Do not segment by individual engineer.
Segments reveal whether the platform has a product-fit problem. If the paved path works for new services but not the revenue-critical legacy estate, “80 percent adoption” is not a comforting answer. It means the teams with the hardest constraints are still carrying bespoke operational cost. That may be reasonable for a while, but it should be an explicit strategy with an owner and a migration decision.
Qualitative evidence matters here. Support tickets, office-hour notes, and brief task interviews can explain why an eligible team did not adopt a tool. Classify the friction: missing capability, unclear documentation, incompatible dependency, reliability concern, or a preference that is unlikely to matter at scale. Then fix the repeated class rather than treating every request as a custom exception.
Protect the measures from their incentives
Platform metrics become dangerous when they are attached to simplistic performance stories. A team that is evaluated on adoption can pressure users into a path before it is ready. A team evaluated on build speed can remove verification that was protecting production. A team evaluated on ticket closure can move confusion into chat threads that never become evidence.
Counter that with paired signals and explicit guardrails. Pair speed with reliability. Pair coverage with successful completion. Pair self-service volume with a small quality sample. Publish the caveat beside the number: data excludes legacy repositories, a release freeze changed deployment volume, or an incident distorted the month. Honest caveats make a metric more useful, not less impressive.
The safe path should eventually become the easy one. That requires defaults, fast feedback, and a supported route that is genuinely less painful than the workaround. The next article in this sequence, Make the Safe Development Path the Easy One, will address those design choices. This review gives that work a place to start.
A health review should change next week
The best platform health review is unglamorous. It identifies a constraint, funds a bounded improvement, and later tells you whether the constraint moved. Engineers should feel the result as fewer mysterious waits, fewer private instructions, and a clearer route through routine work—not as another request to explain their activity.
Keep the scorecard small enough that someone can explain every signal, every caveat, and every active bet. If a chart cannot help you choose what to stop, fix, or fund, leave it off the wall.
