Latency & Barge-in Lab: the thinking.
A proposed measurement rig for where the seconds go in a voice turn, and how cleanly the agent handles being interrupted.
The starting point.
What made this worth building.
Voice agents feel wrong long before they answer wrong. Silence after the caller stops speaking, or an agent that talks over an interruption, breaks the conversation regardless of how good the answer was.
The direction I chose.
The approach I am proposing.
A rig that instruments each stage of a turn — endpoint detection, model first token, speech first byte, network — so the slow stage is identifiable rather than inferred. Alongside it, a barge-in set that interrupts the agent at defined points and checks that it stops, keeps its turn state, and resumes coherently.
How the pieces connect.
Each stage exists because the one before it cannot be trusted on its own.
- 01Instrument the turn
- 02Endpointing, first token, first byte
- 03Budget per stage
- 04Barge-in test set
- 05Interruption behaviour check
- 06Tuning record
The decisions behind it.
The constraints I set deliberately, and would defend.
- Measure per stage, because one end-to-end number hides which component to fix.
- Set a budget per stage and treat exceeding it as a defect rather than a preference.
- Treat barge-in as correctness, not polish — the agent must stop and keep its state.
- Record what each tuning change cost elsewhere, since latency and answer quality trade against each other.
What has to hold true.
How I would know it works — and how I would know it does not.
- Test interruption at the start, middle, and end of an agent turn.
- Test a caller who pauses mid-sentence and expects to keep the turn.
- Compare measured stages against the budget on every configuration change.
- Check that latency tuning has not regressed the behaviour checks in the regression harness.
Where this stands today.
Stated plainly, with nothing implied.
This is a planned concept. Everything above is design reasoning — no part of it has been built or verified yet.
Keep going