Know what the number describes
A precise chart can describe incomplete records perfectly. GitLab analytics uses events from issues, commits, merge requests, environments, deployments, and incidents. When the workflow doesn't create a required record, or creates it with the wrong scope or status, the report answers a different question from the one the reader thinks it answers.
This is especially risky when a data change looks like a performance change. A team that begins recording every production deployment may appear to deploy more often even when its release practice is unchanged. Better incident reporting can raise the reported change failure rate. Two teams with similar delivery paths can look different because one sends external deployments to GitLab and the other doesn't.
Start with the calculation and its source records, not the metric name:
| Measure | Records GitLab uses | What the result doesn't prove |
|---|---|---|
| Deployment frequency | Successful deployments to a production-tier environment, using deployment finish time. | That every successful pipeline reached production. |
| Lead time for changes | Merge request merge time to the successful production deployment that contains the change. | The full time from request or first commit to production. |
| Time to restore service | The time GitLab incidents remain open, with GitLab's documented production-incident and incident-to-deployment assumptions. | That every interruption from another incident system is included or the local incident model fits those assumptions. |
| Change failure rate | GitLab incidents divided by production deployments for the reporting period, using GitLab's daily aggregation. | A direct causal link between each incident and one deployment, or protection from duplicate incident records. |
| Value Stream Analytics stage time | End-event time minus start-event time for items that reach the end event. | The age of work still in progress or the reason time was spent. |
Use the DORA metric calculations1 and Value Stream Analytics stage measurement2 to establish what a result describes. A familiar metric name can conceal a different start event, population, or aggregation from the one your team uses.
Check the records against real work
Keep a short explanation beside a report so another reader can tell which services and work it includes, the reporting period, events and statuses used, units, aggregation, and known gaps. Name who can investigate the data path. Give readers enough information to check the result against real records and judge whether it answers their question.
Then reconcile a sample:
- Select recent work that people can still explain, including an ordinary case and a failed or delayed case when available.
- Follow the records that feed the measure. For a delivery metric, compare the release with its GitLab environment and deployment. For a stability metric, compare known interruptions with GitLab incidents. For a value-stream stage, open the issues or merge requests listed for that stage.
- Check the timestamps, status, links, environment tier, and inclusion rules used by the calculation.
- List missing and duplicate records, work that never reached the end event, and steps performed in another system.
- Compare the sample with the team's written process and with people who perform the work.
- Explain whether the result can inform the intended decision, what limitations apply, and whether more investigation is needed.
The records don't need to be perfect before anyone may view a dashboard. They need to be understood well enough that a reader can tell what the result includes and what it leaves out. A known gap can be acceptable for a local trend and unacceptable for a cross-team comparison. Match the strength of the check to the decision.
Use three kinds of information: GitLab records and settings, the written process or automation, and accounts from people who do the work. When they disagree, record the difference and investigate it. The stated deployment path may omit a manual release step, or a team may follow a workaround that the documentation never captured. Each source answers a different question: what was recorded, what was intended, and what people experienced.
Repeat the sample after changing the branch model, deployment tool, incident integration, environment setup, GitLab version, or reporting scope. Keep a dated note when a correction changes a period that people have already discussed.
Match confidence to the decision
GitLab's DORA reporting draws on different records for each metric. Deployment frequency depends on successful production deployment records. Lead time for changes also depends on merge requests being associated with those deployments. Time to restore service and change failure rate depend on incidents recorded in GitLab. When another tool deploys software, external deployment tracking can create the needed GitLab deployment record. When another system manages incidents, follow the external incident guidance.
Value Stream Analytics measures time between supported events. Aggregated data may take time to appear, and only items that reach the end event are included in the stage calculation. Check the last-updated information before interpreting a recent change.
GitLab reports don't always use the same unit or aggregation. Deployment frequency is shown as a daily average in some views, while the API can return deployment counts for an interval. Lead time and restoration measures commonly use medians. Record the report and interval used instead of referring to “the DORA number” without context.
The stability calculations have important assumptions. GitLab treats tracked incidents as production incidents for these measures. Time to restore service assumes a strict one-to-one relationship between incidents and deployments. Change failure rate doesn't link each incident directly to a deployment, and duplicate incidents can change the result. Document the incident population and any change in reporting practice beside the trend.
Begin with one service or a set of services that share a delivery model. A continuously delivered service, a scheduled mobile release, and a library publication can all be healthy while producing very different patterns. Compare teams only after their records, definitions, and work are similar enough for the comparison to guide a decision.
Questions to ask
- What decision will this result inform, and who will make it?
- Which service, project, work type, and period does it cover?
- Which event starts the calculation, and which event ends it?
- What status, relationship, environment tier, and aggregation qualify a record?
- Which real releases or interruptions are missing or duplicated in GitLab?
- Which deployments or incidents begin in another system, and how do they reach GitLab?
- Which work is absent because it never reached a stage's end event?
- Where do the records, written process, and team account disagree?
- Has the report finished processing recent events?
- Can the result inform this decision? What limitations apply, or is more investigation needed?
- Who repairs the data path and repeats the check after a workflow change?
Further Reading
Notes
-
GitLab's DORA reporting is available in Ultimate. Teams can still measure delivery outcomes using records from their existing tools; the tier provides GitLab's built-in calculations and views. ↩
-
Value Stream Analytics feature availability differs by tier and scope. Free provides project-level default stages. Premium and Ultimate add group views, aggregation, and custom value streams and stages. ↩
- DORA metrics helps you check a report's calculation, units, and incident assumptions against the decision you need to make.
- Value stream analytics explains how to inspect stage items, choose supported events, and check filters and processing delays.
- Environments and deployments shows how to connect deployment jobs to the environments whose releases you want to measure.
- Track deployments of an external deployment tool explains how to send release outcomes to GitLab when another system deploys the software.
- Incidents helps you establish the opening and closure workflow behind stability reporting.
- Reliable DORA and value-stream data provides a sequence for checking deployment, merge request, incident, and stage records before expanding reporting.