# Quality New to DORA? Start with [What is DORA?](/docs/engineering-delivery/what-is-dora), then come back here. The Quality page shows how often your releases need fixing and how much of the team's time goes into repair rather than building new things. You find it under **Delivery**, as the **Quality** tab. This guide explains everything on the page, what each reading means and what to do about it. ## What the Quality page shows At the top, the **Quality update** says how many deploys needed a fix and which service has the highest share, for example "8% of deploys needed a fix, down 2 points. Payments has the highest share, 2 of 6 deploys." With no code source connected it says "Connect your code to see whether deploys stick. Nothing is estimated." The tiles are **Change fail rate**, **Deployment rework rate**, **Bugs opened** and **Reopened tickets**, each explained below. Under the tiles, the page shows your deploys to production in the period next to the **changes per deploy**, the number of merged changes a typical deploy carries: "Smaller batches are easier to undo." Below that, **Change fail rate, by service** shows "Which services' deploys need a fix". The first two are DORA metrics, read from deploys and incidents. The last two come from your work tracker and catch problems found later. Quality is read per service where it can be, because one fragile service can hide inside a team average. ## Change fail rate Change fail rate is the share of deploys that were rolled back or hotfixed, over all deploys in the period you choose. It tells you how often a release causes trouble straight away. A planned follow-up change is not a failure, and a problem caught before production is your safety net working, not a failure. The tile compares the period with the one before. When it rises, open it for the releases that needed a fix and look for what they share. [Change fail rate](/docs/engineering-delivery/change-fail-rate) explains the metric in full, including DORA's figures and how to lower it. ## Change fail rate by service **Which services' deploys need a fix** shows each service's share of deploys that were rolled back or hotfixed, highest first, "with the deploys each rests on. Each service against its own past, no outside grade." ### How to read it Always read the rate next to the number of deploys it rests on. A service with 4 deploys jumps from 0% to 25% with one failure; a service with 40 deploys and the same 25% has a real pattern. Compare each service with its own past, not with other services or teams. ### What to do about the top service 1. Open the service's bar to see its releases that needed a fix. 2. Take the last few and look, without blame, for what they share: size, a missing test, a configuration change, one fragile area of code. 3. Add an automated check for the most recent failure, so the same problem cannot ship twice. 4. Look at the size of the changes behind its fixes. Large changes break more often; see [Change size](/docs/engineering-delivery/review#change-size). 5. Resist adding another approval step. DORA found heavyweight approvals did not lower change fail rates and made delivery slower. When one service needs fixes at least 5 percentage points more often than every other, the Overview's Worth a look calls it out, and the alert **One service needs fixes more often** can tell the people who see its team. [Change fail rate by service](/docs/engineering-delivery/change-fail-rate#change-fail-rate-by-service) explains why DORA asks for this view. ## Deployment rework rate Deployment rework rate is the share of deploys that were unplanned and shipped because of an incident in production. Where change fail rate counts the release that broke, rework rate counts the emergency deploys that followed. The app reads it from the incident source you choose in Settings. A deploy counts as unplanned when it shipped while an incident was open, to the service the incident names, or to a service of its team when it names none. Only a resolved incident links deploys, so one left open by mistake never marks every deploy after it. Without an incident source, the tile is not measured, and never estimated. Just after you connect one, it says "Your incident source is {source}. Its incidents arrive after the next pipeline run, and rework rate is measured from then. Nothing is estimated." [Deployment rework rate](/docs/engineering-delivery/deployment-rework-rate) explains the metric in full, and [Incidents](/docs/engineering-delivery/incidents) covers the incident source. ## Beyond the deployment metrics Change fail rate and rework rate look at the moment of deployment. The two signals below, bugs opened and reopened tickets, add the view from your work tracker: problems found after release, and work that was not really finished. The DORA metrics catch failures that happen when a change is deployed, but many quality problems show up later, as bug reports and work that comes back. DORA's two instability metrics are [change fail rate](/docs/engineering-delivery/change-fail-rate), the share of deployments that need immediate fixing, and [deployment rework rate](/docs/engineering-delivery/deployment-rework-rate), the share of deployments that were unplanned and made because of a production incident. Both are about deployments. A bug a customer reports three weeks after release, or a ticket that was marked done and had to be reopened, does not show up in either. That is where tracker signals help. They are less precise than the DORA metrics, and none of them has a published benchmark, but together they tell you whether quality problems are growing and how much of the team's capacity they take. Throughout this page we follow a fictional team of six who build an online booking app and track their work in Jira. ### Failure demand Failure demand is work a team has to do only because something earlier went wrong: fixing bugs, redoing work that was not right the first time, answering support requests caused by a defect. The [glossary](/docs/engineering-delivery/glossary#failure-demand) has a short definition. It is the opposite of value demand, the new things customers actually want. Every hour spent on failure demand is an hour not spent building, so a team that wants to deliver more often gains most by preventing failures rather than by working faster. Each signal below measures one side of failure demand. > **Note:** These signals describe the team's work, not anyone's performance. A bug belongs to the system that let it through, not to the person who wrote the line. ## Bugs opened Bugs opened is the number of bugs reported in a period, read against the number resolved in the same period. ### Why it matters Bugs opened against bugs resolved says whether quality debt is growing. If more bugs arrive than the team fixes, the backlog of known problems gets longer. ### How to read it Compare the two numbers, and the trend, rather than looking at bugs opened alone. In the booking team, 18 bugs were opened last month and 15 were resolved. Three more arrived than were fixed, which is worth watching but not alarming on its own. Be careful with a rise. More bugs opened can mean quality got worse, but it can also mean more people are using the product, or that a new support process makes reporting easier. A rise is a question, not a verdict. Look at what the new bugs have in common before drawing conclusions. ### What good looks like No published source gives a figure. Compare with your own past. ### How to improve it - Look at the bugs opened in the period for a shared cause: one service, one feature, one kind of mistake. - Connect it to the deployment view. If bugs cluster around one service, check its [change fail rate by service](/docs/engineering-delivery/change-fail-rate#change-fail-rate-by-service). - Fix the cause, not just the instance. Add an automated test for the kind of failure you keep seeing; DORA's [test automation](https://dora.dev/capabilities/test-automation/) capability is the long-term answer. - Keep changes small. Smaller changes are easier to test properly and less likely to introduce bugs. See [Change size](/docs/engineering-delivery/review#change-size). ### Common mistakes - Treating a drop in bug reports as good news without checking whether people simply stopped reporting. - Closing old bugs in bulk to make the numbers look better. The problems are still there. ### How Solidafy measures it The bugs opened in the period you choose, by Jira's issue type or a Linear label that names a bug, against those resolved in it. A bug closed as canceled, a duplicate for example, is left out, and so are Jira issues reported by an app account. ## Reopened tickets Reopened tickets is the share of finished tickets that were moved out of done and back into work at least once. ### Why it matters A ticket moved back out of done was not finished the first time. It is the tracker's side of [rework](/docs/engineering-delivery/glossary#rework): the team thought the work was complete, moved on, and then had to come back to it. That switch costs time twice, once to pick the work back up and once for whatever the new work displaced. ### How to read it In the booking team, 6 of the 52 tickets finished last month had been reopened, about 12%. Open them and look for a pattern. Common ones: - The work met the ticket's words but not what was actually needed. - Testing happened after the ticket was marked done, not before. - A fix covered one case and missed a closely related one. ### What good looks like No published source gives a figure. Compare with your own past. ### How to improve it - Open the reopened tickets and look for what was missed before done. - Agree as a team what "done" means, for example tested and deployed, not just merged. - Talk to the person who asked for the work before you start, not only after. ### Common mistakes - Creating a new ticket instead of reopening the old one, to keep the numbers clean. It hides exactly the signal you want. - Counting a ticket closed as canceled as a reopen. It is not rework. ### How Solidafy measures it The share of tickets finished in the period you choose that the tracker's status history shows moved out of done back into work at least once. Closing a done ticket as canceled is not a reopen, and a ticket whose history holds no move is left out. Jira issues reported by an app account are left out too. ## How these fit with the DORA metrics Read the tracker signals next to the deployment metrics, because each sees a different moment. - **At deployment:** [change fail rate](/docs/engineering-delivery/change-fail-rate) tells you how often a release needed fixing straight away, and [deployment rework rate](/docs/engineering-delivery/deployment-rework-rate) how often you had to ship an unplanned deploy because of an incident. - **After release:** bugs opened tells you how many problems people found later. - **Inside the team:** reopened tickets tells you how often work was not finished the first time. If change fail rate is low but bugs opened keeps rising, problems are slipping past your checks and reaching customers quietly. If all of them rise together, quality is the constraint to work on next. Solidafy's Quality section shows these side by side, reads quality per service where it can, and compares every figure with your own past rather than an outside grade. ## Questions the app raises ### Change fail rate went up. What should we do first? Open **Which services' deploys need a fix** and start with the service at the top that rests on a fair number of deploys. Take its last few releases that needed a fix and look for a shared cause. Most rises come from one service or one kind of change, not from everything getting worse at once. If the team wants to make it a priority, set "Safer releases" as the focus on the [Overview](/docs/engineering-delivery/overview-and-focus). ### One service shows 50% but only has a couple of deploys. Should we worry? Not yet. With two deploys, one failure is 50%. Watch it over a longer period, or widen the period you are looking at, before acting. The deploy count next to each bar is there for exactly this. ### Why is deployment rework rate not measured? It needs incidents, and incidents come only from an incident source: Jira, Linear or PagerDuty. Choose one under **Incident source** in Settings. The app never guesses which deploys were unplanned. ### Bugs opened went up. Is quality getting worse? Maybe, but a rise can also mean more people use the product or reporting got easier. Compare it with bugs resolved, then open the bugs for a shared cause: one service, one feature, one kind of mistake. If change fail rate is flat while bugs rise, problems may be slipping past your checks and reaching customers later. ### Many tickets are reopened. What does that tell us? Work is being marked done before it really is. Open the reopened tickets and look for the pattern: testing after done, a fix that missed a related case, or work that met the ticket's words but not what was needed. Agree as a team what done means. ### A tile says it is not measured. Why? Change fail rate needs a code source that shows deploys, rework rate needs an incident source, and the bug and reopened tiles need a work tracker. See [Setting it up](/docs/engineering-delivery/setup#why-does-a-tile-say-it-is-not-measured). ## Learn more Official sources: - [DORA's software delivery performance metrics](https://dora.dev/guides/dora-metrics/) - [DORA capability: test automation](https://dora.dev/capabilities/test-automation/) - [DORA capability: working in small batches](https://dora.dev/capabilities/working-in-small-batches/) Related Solidafy guides: - [Change fail rate](/docs/engineering-delivery/change-fail-rate) - [Deployment rework rate](/docs/engineering-delivery/deployment-rework-rate) - [Code review speed and size](/docs/engineering-delivery/review) - [Flow of work](/docs/engineering-delivery/flow) - [Improving software delivery](/docs/engineering-delivery/overview-and-focus) - [Glossary](/docs/engineering-delivery/glossary)