Operate a daily reconciliation queue for interactions that should have reached QA but have no usable score. Assign each missing record a reason, an owner and a next action. Keep delayed, excluded, duplicate and failed records separate so teams can repair the pipeline without hiding the gap.
Level AI describes evaluation, calibration and dispute handling in its QA product. AWS documents exporting contact records to Kinesis Data Streams. AWS documents producer and consumer retries that can deliver duplicate records. These are product-specific public descriptions, not results of our own product testing. Confirm your required configuration, evidence access and operating responsibilities in a scoped demonstration.
How should the daily balance prove that nothing disappeared?
Use an auditable identity balance rather than comparing two dashboard totals. The following invented example assumes a single frozen source period and distinct contact identifiers.
| Disposition at cutoff | Illustrative unique contacts |
|---|---|
| Source population | 100 |
| Accepted documented exclusions | 10 |
| Eligible population | 90 |
| Valid scores linked to source identifiers | 82 |
| Open pipeline exceptions | 6 |
| Confirmed source loss, still counted as coverage loss | 2 |
The eligible balance is 90 = 82 + 6 + 2. Retrying one exception twice does not add two contacts; record attempts under the original identifier. When a late result arrives, move the contact between states while preserving the earlier cutoff snapshot. Our recommended closure log records the previous state, new state, timestamp, owner and authoritative score identifier. Publish open exceptions and confirmed loss separately: administrative closure of an unrecoverable record must never turn a missing score into a success.
What is the daily reconciliation boundary?
Agree a reporting cutoff and an expected processing delay with the pipeline owner. Export identifiers for the source period and compare them with QA results using stable interaction identifiers. Recheck late arrivals in a later run rather than moving the period boundary until the numbers look complete.
Define whether a transfer creates a new contact identifier or another segment of the same interaction. Retain the source identity and any platform identity together. Matching only by timestamp or agent name can merge unrelated contacts or make one missing record appear repaired by another.
What should an exception record contain?
| Field | Operating purpose |
|---|---|
| Source and QA identifiers | Locate the same interaction across systems |
| Expected and last-seen stage | Identify where progress stopped |
| Reason and supporting evidence | Separate failure from documented exclusion |
| Owner, next action and due time | Prevent unowned recurring exceptions |
| Attempt and resolution history | Avoid repeated work and silent disappearance |
How should the queue be triaged?
Start by checking whether the source recording or text exists. If it does, inspect capture, transcription, eligibility and scoring in order. Assign the issue to the owner of the failed stage. A missing transcript and a criterion marked not applicable are different states, even if neither produces an ordinary numeric result.
Before replaying data, confirm duplicate behavior with the implementation team. A retry should not silently create multiple evaluations that inflate volume or change an agent average. Preserve attempt history and explicitly identify the authoritative final record. Use synthetic or approved test cases when validating recovery behavior.
When is an exception resolved?
- A valid score is linked to the original source identity.
- A documented exclusion is accepted by the responsible owner.
- A confirmed source-data loss is recorded as unresolved coverage loss.
- A duplicate is linked to its authoritative record without double counting.
Keep aging and recurring causes visible in a daily summary. A record should not be considered resolved merely because it falls outside the default dashboard date range. Use reporting denominator checks when publishing results after a repair.
Limits
This queue design is an editorial recommendation, not a vendor integration specification. Sources were checked on 20 October 2026. No ingestion pipeline, export schema or retry was tested. Identity semantics, processing delays and duplicate handling vary by implementation; confirm them before replaying production records. Validate replay behavior with the pipeline owner before operating it.