A team can spend hours maintaining a dashboard and still disagree about what to change. More charts won't resolve that disagreement if nobody has decided which question the numbers should answer.
Start with a decision the team can make, then choose a few measures that help with it. You might want to shorten the wait for review, get test results to developers sooner, or find out whether a new release process reduces disruption. The aim is to understand whether work is getting better, not to make every number go up.
Start with a decision someone can act on
Ask the people responsible for the work what they would do differently with better information. “Track engineering productivity” doesn't identify a decision. “Decide whether to add runner capacity or shorten the tests” does: queue time and execution time point to different changes.
Agree on who can make that choice and which work you'll examine. Start with one service or a set of projects that follow a similar delivery process. A group-wide average can hide the difference between an application deployed every day and one released on a fixed schedule.
If nobody can explain how a report might change a decision, set it aside. You don't need to collect every available metric before you can improve one part of delivery.
Choose a main result and check for unwanted effects
Pick a measure close to the problem. Then ask what could get worse while that measure improves. The answer tells you what to review alongside it.
| Decision | Main result to examine | What to check alongside it |
|---|---|---|
| Whether to change reviewer assignment | Wait until review begins | Review quality, reviewer workload, and time through to merge |
| Whether to reorganize pipeline jobs | Time until developers receive feedback they can act on | Retries, missed checks, runner cost, and total pipeline time |
| Whether to release in smaller batches | Time a completed change waits for release | Failed releases, recovery effort, and disruption to users |
These are starting points, not a standard scorecard. Choose checks that could change your decision, and leave out measures that don't help explain the result.
Activity counts need particular care. A rise in merge requests might reflect smaller changes, duplicated work, or a different way of splitting tasks. It doesn't establish that customers received more value. Avoid individual targets for commits, lines of code, or merge requests; those targets encourage visible activity rather than better delivery.
The same distinction matters when agents carry work across several stages. A faster implementation may leave review waiting longer or move correction into operations. Follow the work through to the intended outcome and include the effort it created elsewhere. If the goal is quicker recovery for customers, neither more agent sessions nor more merged fixes answers whether customers recovered sooner.
Consumption belongs on the cost side of that comparison, alongside review, rework, infrastructure, and ongoing support. Cost per accepted change can help assess delivery efficiency, but it doesn't establish the return from that change. Evaluating business value and the cost of delivery explains how to assess an AI investment against the result the organization wanted.
Establish a starting point you can compare with
Before changing the workflow, inspect recent examples and record what happened. Include enough normal work to see its variation, not just a quiet week or a particularly bad release. For infrequent releases, choose a review point based on completed releases rather than assuming a week will tell you enough.
Keep the starting point with the improvement issue: which projects and types of work it includes, the period, how you calculated the result, and anything missing. For a percentage, write down what counts in both parts of the calculation. “Most releases passed” tells a different story if failed releases were left out of the total.
Check the records behind a few results before relying on the chart. GitLab's DORA reporting can help with delivery trends,1 but a metric's name may cover less than your question. Lead time for changes starts at merge, so it won't explain a long wait for review before merging. Reliable DORA and value-stream data explains how to check those inputs and choose an appropriate interpretation.
Keep open and blocked work in view too. A team can finish small tasks faster while its oldest work continues to wait. Look at individual examples as well as the typical duration, and ask whether a change in workload or recording practice explains the difference.
Try one change and listen to the people using it
Choose an improvement small enough to learn from without changing the whole process at once. Give someone time to own it, agree when you'll review it, and decide how you'll undo it if the effects are unwelcome. A value-stream improvement cycle shows how to trace a delay and choose that first trial.
For example, suppose developers wait a long time for pipeline feedback. A team moves fast checks earlier without removing the slower tests. It compares the wait for the first actionable result before and after the change, then checks whether retries or runner costs increased. If feedback arrives sooner but developers still have to rerun jobs to understand a failure, the timing alone overstates the improvement.
Ask about recent work: What became easier? What still interrupts you? Did a workaround disappear, or just move to someone else? Explain how you'll use the answers and offer individual conversations when people wouldn't speak freely in a group. Summarize the problem without turning personal feedback into a performance record.
Decide what the results warrant
At the review point, bring the measures and examples back to the people involved. A change is worth keeping when it helps the intended work and its costs are acceptable—not simply when one chart improves.
When results conflict, investigate before expanding. Faster review with more post-merge corrections may call for a different review approach. Lower recorded incident duration may reflect a changed closure process rather than faster recovery. A trial with too little comparable work may need more time; missing events may need an integration fix instead.
Record the decision and its reason in the existing improvement issue. Keep a change that works, adjust one that partly helps, or stop one whose costs outweigh its benefit. Let participants know what their feedback changed. When expanding, begin with teams whose work resembles the trial and check the result again as the delivery process becomes more varied.
Stop measuring what no longer guides a decision
Review the reporting effort as well as the improvement. Once a problem is resolved, you may need only a periodic check that it hasn't returned. A manually maintained weekly report shouldn't become permanent just because it helped during the trial.
Change or retire measures when the team can no longer act on them, when their definitions no longer fit the workflow, or when collecting them costs more than the decisions they support. Keep enough history to explain the change so a new reporting method isn't mistaken for better or worse performance.
Further Reading
Notes
-
GitLab's DORA reporting is available in Ultimate. Teams on other tiers can work with their existing deployment and incident records; each measure still needs a clear definition and consistent recording. ↩
- DORA metrics explains which events GitLab uses and what each calculation includes.
- A value-stream improvement cycle helps you investigate a delay before choosing a change.
- Reliable DORA and value-stream data helps you distinguish a change in delivery from a change in the records behind the report.