Concept
Quality scores
Updated
On this page
Scores turns your company's own results into scores from 0 to 100: the test runs you ran, and the decisions Zof made on your pull requests. Every score is a share of real results over the last 30 days. Zof never fills a gap with a guess: a score with too little behind it says Not enough data yet, never 0 or 100.
Where do the scores come from?
Two sources, read when you open the page. Back on Scores or Team scores within 30 seconds, you see the same answer; after that, the last answer stays on screen while Zof reads again, and a failed read shows as unavailable, never as the old scores:
- Test runs: the runs you start on the Run page or on a schedule, the same runs Reports counts. Pull request checks aren't test runs here; they count through the decisions below.
- Zof decisions: the decision Zof made on each pull request, the same decisions Releases shows.
A run counts on the day it started, as the word its run page gives it: an engine pass the run page calls Couldn't check is not a pass here. A run that is still going, or was cancelled, isn't counted. A finished run Zof hasn't read yet isn't counted either, and the page says how many.
How is each score calculated?
Test categories
Functional Quality, Security Posture, Performance and API Health each score the share of their finished runs that passed:
score = passed ÷ (passed + failed)
A run Zof couldn't check, for example because the address didn't answer, is neither a pass nor a fail here. It counts in Reliability instead.
A score needs at least 5 results behind it: 5 runs that passed or failed, 5 finished runs for Reliability, 5 decisions for Release Gates. With fewer, the category says Not enough data yet and how many more it needs. The factors below it still show their counts.
| Category | Test types in it |
|---|---|
| Functional Quality | Smoke, end-to-end, unit, regression, accessibility and payment idempotency tests |
| Security Posture | Transport security (HTTPS/TLS), SQL injection and compliance scans |
| Performance | Latency, load and endurance tests |
| API Health | API contract tests |
Each category lists one factor per test type that ran, with its own pass rate. See What each test checks for what a pass means for each.
Reliability
The share of finished runs that ran to a result:
score = (passed + failed) ÷ (passed + failed + couldn't check)
Every test run counts here, whatever its type.
Release Gates
The share of the listed Zof decisions evaluated in the last 30 days that allowed the change:
score = allowed ÷ decided
A decision that denied the change, or that needs an approval or evidence first, counts as decided but not allowed. A pull request Zof hasn't decided yet isn't counted, and a pull request Zof evaluated more than once counts once, in the window of its latest evaluation. See How Zof decides.
This score is always Partial: the pull request list is selected by when code last changed, not when Zof evaluated it. Older closed pull requests can be absent even if Zof evaluated them recently, and the list has a size limit. The score describes the decisions returned, not all decisions in the period.
Overall readiness score
The plain average of the scored categories, with equal weights; an unscored one shows Not scored. It needs at least 3 scored categories, one of them from your own test runs: Release Gates alone, whose decisions may predate your company, never makes one. With fewer, it says Not enough data yet.
The top of Scores shows it and its change from last week, with each category below. It takes a colour only when all 6 categories are scored: an average of some of them is a number, not a verdict.
Trend
The arrow beside a score compares it with the same score 7 days earlier, over that day's own 30-day window. With no earlier score to compare, there is no arrow. The change in the overall score compares only the categories scored in both weeks, so a category that starts or stops being scored doesn't move it on its own. The bars under the overall score are the overall score of each of the last 14 days, when the sources cover those windows.
Zof's list of decisions can't establish a complete decision history, so Release Gates has no trend. While that list contributes to scores, the overall change and daily bars are also unavailable. Test-category and Reliability trends still work when the run history covers their windows.
What isn't scored?
Some factors on the page have no source in Zof yet: Uptime, Flaky Test Rate, Recovery Time, Resource Utilization, Error Rate, Dependency Health and Secret Detection. Each shows NOT AVAILABLE with the reason, and none of them counts toward any score.
A category with fewer than 5 results in the last 30 days has no score. If nothing at all can be scored yet, the page says so.
What does Partial mean?
When a source can't be read, or can be read only in part, the page says Partial and lists what's missing. An unavailable source's scores show — with the reason. Scores from incomplete sources use only the results returned, with the limitation disclosed.
- Runs couldn't be read: the test categories and Reliability have no score.
- Based on your newest runs only: Zof reads up to 500 runs; past that, older runs in the window weren't read.
- Finished runs haven't been read yet: Zof reads finished runs a batch at a time. Those not read yet aren't counted. The page says when a reload can read more.
- Zof's decisions couldn't be read: Release Gates has no score.
- Release Gates uses only listed pull requests: the list may omit decisions evaluated in the last 30 days, even when it hasn't reached its size limit. Release Gates is based on the ones listed; its trend and the overall history are unavailable.
When a source was read only in part, the trend and the daily bars show no score for any window that part may not cover.
If GitHub isn't connected, Release Gates has no decisions to score and says so. That isn't Partial: there's nothing missing. If runs can't be read either, there's nothing to score, and the page says scores are unavailable, with Retry.
Team scores
Team scores lists each person who started a test run in the last 30 days, with the pass rate of the runs they started (passed ÷ (passed + failed)), shown from 5 such runs. Scheduled runs are listed as Schedules. Someone no longer in your company shows as a former member.
This measures the runs, not the person or their code: a run counts for whoever started it. Zof decisions aren't tied to a person, so they aren't part of team scores.
Who can see scores?
Everyone in your company, viewers included. Scores only read your results; nothing on these pages changes anything.
Sharing a scorecard
Members and admins see Share on Scores. It makes a link to today's scorecard: the overall score, each category with its factors, the daily history and any Partial notes, frozen as they are when you share. Later changes aren't shown. Team scores are never shared, so nobody's name leaves your company through a scorecard link. While a source can't be read, no link is made and Share asks you to try again in a minute. Viewers don't see Share.