Publish QA scores with the calculation, applicable question count and evaluation population shown together. Separate not applicable from missing evidence and technical failure. A change in weights, answer eligibility or contact mix can move the headline score even when the underlying behavior does not change.
Level AI describes scorecard performance dashboards. AWS documents evaluation averages with version-specific weights and calibration exclusions. Check the exact metric definition in the system you use. The worked example below is an illustrative calculation, not a claim about either product’s default scoring implementation.
Which denominator answers your question?
A denominator is the count or total possible weight beneath a reported rate. Distinguish the number of contacts, submitted evaluations, applicable questions and successfully processed inputs. One contact can have multiple evaluations; one evaluation can have several not-applicable questions. An average of percentages is not automatically equivalent to the percentage calculated from pooled questions.
How can the same results produce two different percentages?
| Illustrative contact | Passed applicable questions | Contact score |
|---|---|---|
| A | 1 of 2 | 50% |
| B | 1 of 1 | 100% |
If contacts receive equal weight, the mean of their scores is 75%: add 50 and 100, then divide by two contacts. If applicable questions receive equal weight, the pooled pass rate is 66.7%: two passed questions divided by three applicable questions. Neither number is inherently wrong. They answer different questions and must be labeled accordingly.
Unequal criterion weights create another calculation. Write the formula and test it against exported rows rather than guessing from the dashboard label. Preserve the weight version used for each evaluation. Do not silently apply today’s weights to older results and present the resulting series as unchanged history.
How should not-applicable and missing results be treated?
- Not applicable: the documented trigger for a criterion did not occur.
- Insufficient evidence: the trigger or behavior cannot be established from the record.
- Technical failure: processing did not produce a usable result.
- Excluded evaluation: a documented report rule removes the record from this calculation.
Our recommendation is to show these categories separately from pass and fail counts. If a platform collapses them into one field, retain an additional reconciliation record before drawing operational conclusions. A rising score alongside a rising missing-evidence count deserves investigation, not an immediate success claim.
What should a reporting release check include?
Recalculate a small known set containing different question counts, partial answers, rescored evaluations and changed weights. Check whether reports use interaction time or evaluation submission time. Confirm which evaluation is authoritative after an appeal and whether calibration work is excluded. Record the chosen rules in the metric dictionary.
See metric dictionary design for ownership and daily record reconciliation for technical gaps.
What reconciliation should accompany a reported improvement?
Use a small release fixture before comparing scorecard versions. In an illustrative 100-contact batch, 80 contacts receive a usable evaluation, 12 cannot be processed and eight remain pending. If 72 of the evaluated contacts meet the defined standard, report 72/80, or 90%, for evaluated contacts and 80/100, or 80%, for coverage. Do not report 90% as the quality of all 100 contacts.
Now suppose a new scorecard makes a difficult question inapplicable on 20 contacts. Preserve the old results and show how the eligible population changed. A higher new score could reflect a justified scope correction or a rule that hides difficult work; the score alone cannot decide which explanation applies.
| Reconciliation field | Release check |
|---|---|
| Population identity | Same contact IDs, duplicates and reporting window accounted for |
| Question eligibility | Reason for each not-applicable result remains inspectable |
| Weight and appeal version | Both original and authoritative revised scores retained |
| Coverage difference | Failed and pending inputs shown independently of quality |
This proposed fixture tests arithmetic and report interpretation. It does not validate whether the underlying QA judgments are correct.
Limits
This article supplies transparent arithmetic and reporting recommendations, not a universal vendor calculation. Sources were checked on 20 October 2026. We did not inspect a customer export or dashboard implementation. Eligibility, versioning and rescoring behavior require direct verification; a numeric average does not establish fairness, accuracy or a causal service improvement.