Expand description
What the listener heard of the model’s speech.
The model streams its audio faster than it plays, and its transcript runs further ahead still. Measured on the Live API, output transcription arrives up to twice as far into the answer as the audio delivered with it. When the listener barges in, the model’s transcript therefore holds words the listener never heard: a confirmation cut off halfway, or a verbatim disclosure that was never finished.
Whatever plays the audio reports to the session’s PlaybackClock
(LiveHandle::playback). It reports the
audio it queues and the moment it flushes on barge-in. The voice pump,
and so every telephony bridge built on it, reports automatically. When
the model is interrupted, the runtime reads the clock and cuts the
model’s side of the turn to what was heard. That cut applies to the
transcript buffer, the final on_output_transcript callback and the
verbatim check. Heard audio becomes text through the session’s speaking
rate, which is calibrated on its uninterrupted turns (16 characters a
second until then, which matches English speech on current Live models).
The cut ends at the last whole word.
A session with no playback reporter cuts to the audio it received, which is an upper bound on what could have been heard. A turn with no audio, as in a text session, is not cut.
The model’s own context is not changed. The Live API keeps what it sent, so the model may still believe it finished its sentence.
Structs§
- Playback
Clock - Playback progress of the model’s audio, reported by whatever plays it. See the module docs.