Gemini 3.8 Live

Gemini 3.8 Live (gemini-3.8-live, GA since 2026-09-24) is the current real-time model, on both Vertex AI and Google AI (Gemini API key). Compared with Gemini 2.5 Flash Live Native Audio it adds custom transcription vocabulary, blocking tool calls that the server cancels when the user speaks again, polite handling of INTERRUPT-scheduled tool responses, and, on Vertex AI, Live Avatar video. Affective dialogue and proactive audio are always on, and thinking is not supported. A separate model, gemini-3.8-live-extended-thinking, does think (see below).

Select it with ModelId::LIVE_3_8. The platform default model is unchanged, so existing sessions keep their current model until you opt in.

use gemini_adk_fluent_rs::prelude::*;

let handle = Live::builder()
    .model(ModelId::LIVE_3_8)
    .instruction(
        "You are an enterprise concierge. When a tool returns no results, tell \
         the user before calling it again. Never issue more than two \
         consecutive tool calls without speaking.",
    )
    .custom_vocabulary(["ORD-8472", "Express Shipping"])
    .transcription()
    .connect_from_env()
    .await?;

What the SDK does for you

The setup message is shaped per model (LiveModelProfile, keyed on the model name) and per platform. Everything below was measured against the live Google AI endpoint on 2026-09-25. A “1007” is the close code the server sends when it refuses a setup or a message.

Setting	Gemini 3.8 Live on the wire	Why
.thinking(..), .include_thoughts(), .thinking_level(..)	left off	The model has no thinking. A thinkingLevel is refused (1007), and a budget has no effect.
.affective_dialog()	left off	Always on. Sending enableAffectiveDialog closes the session on the first input (1007).
.proactive_audio()	left off	Always on, and Google AI has no proactivity field for any model (1007 at setup).
Tool behavior (BLOCKING / NON_BLOCKING) and response scheduling	sent	Earlier Vertex AI Live models reject both, so they are still stripped there.

Google AI also has no explicitVadSignal, no sessionResumption.transparent, and no avatarConfig.avatarName / customizedAvatar. Each of these is refused at setup, so the SDK leaves them off on Google AI.

Anything left off is listed by SessionConfig::ignored_settings(), and connecting logs it once as a warning. A setting without effect is visible, not silent. Models the SDK has no profile for get every field passed through.

Changing instructions mid-session

update_instruction(..), which the default InstructionUpdate steering mode uses on phase transitions, is sent differently per platform:

• Vertex AI: a system-role client content turn, as Vertex AI documents.
• Google AI: a user-role turn that says it replaces the instructions, with turnComplete: false. A system role closes the session on Google AI (1007); this was measured on Gemini 2.5, 3.1 and 3.8 Live. Gemini 3.1 and 3.8 follow the update on the next turn. Gemini 2.5 accepts it without following it: for a persona change on 2.5, use ContextInjection steering or start a new session.

Context injected mid-session as client content with turnComplete: false (ContextInjection steering, per-turn modifiers) is accepted by 3.8 without triggering speech, and the model uses it. So is a tool response scheduled SILENT: it adds its result to the context without the model speaking.

Replacing tools and the instruction mid-session

SessionHandle::update_context sends a contextUpdate message. It replaces the declared tools, the system instruction, or both, without starting a new session:

use gemini_genai_rs::prelude::*;

session
    .update_context(
        ContextUpdate::new()
            .system_instruction("The caller is verified. Read balances with play_value.")
            .tools(vec![Tool::functions(servicing_tools)]),
    )
    .await?;

Only the fields you set change. tools(vec![]) clears every tool, and an update with nothing set sends nothing. Updates are processed in order with the rest of your input. So an update sent just before a tool response is in effect when the model reads that response, and the response can point the model at a tool the update has just declared.

Each update invalidates the server’s prefix cache, so send one when the tool set or the instruction actually changes, not on every turn. After an update, a reconnect declares the updated tools and instruction in its setup message, not the ones the session started with. A resume ignores those declarations: the server restores the session it holds, which lacks an update whose frame was lost with the connection. So after a resumed setup, the session sends the fields that updates have replaced once more.

Measured on Google AI on 2026-10-09 with examples/context-update-spike (three runs, same results each time):

Model	Result
Gemini 3.8 Live	Applied. The model called a tool that only the update declared, said it had no tool for one the update removed, and followed a replaced instruction on the next turn.
Gemini 3.8 Live Extended Thinking	Applied, as above.
Gemini 2.5 native audio	Refused: the session closes with 1007, “Request contains an invalid argument”.

With Gemini 3.8 Live, an update sent while a tool call was pending was accepted. The tool response then named the next step, and the model called the tool the update had just declared. An update sent while the model was speaking did not interrupt it. After a resume whose setup declared the old tools, the updated tools were still in effect. The reverse also held: when the first connection never sent the update and the resume’s setup declared the new tools, the model said it had no such tool until the update was sent again after the resumed setup. Shrinking the declarations from twelve tools to one cut the prompt from about 1,650 to about 700 tokens per turn. Extended Thinking reported no such drop.

An update does not change a reply the model is already forming. With --only discovery (five runs each), a tool declared before the question was used in that turn’s reply 5 of 5 times; declared 0, 300 or 800 ms after it, with no call pending, 0 of 5 each. A pending call is the exception: declared before the response to a load_capability call the model made, the tool was used 5 of 5 times, at about 800 ms instead of 340 ms.

To have each phase or flow step declare its own tools, set SteeringMode::ContextUpdate and let the runtime send these updates; see steering modes.

SessionConfig::supports_context_update() reports whether the configured model accepts the message. Where it does not, the session reports a codec error event and stays up rather than sending a message that would close it. Vertex AI defines the same message; the measurements above are from Google AI.

Live Avatar (Vertex AI)

.avatar(..) makes the model answer with synchronized 24 FPS video and sets the response modality to VIDEO, as the API requires. Video arrives on on_media (L2), EventCallbacks::on_media (L1), LiveEvent::Media, or SessionEvent::Media (L0), as InlineMedia { mime_type, data } chunks (video/mp4). Speech still arrives on on_audio. Both are suppressed after a barge-in.

Live::builder()
    .model(ModelId::LIVE_3_8)
    .voice(Voice::Puck)
    .avatar(AvatarConfig::prebuilt("Ben"))
    .on_media(|chunk| renderer.push(chunk.data.clone()))
    .on_audio(|pcm| speaker.push(pcm.clone()));

Avatars are a Vertex AI feature. On Google AI, gemini-3.8-live refuses the VIDEO modality (“The requested combination of response modalities (VIDEO) is not supported by the model”), and its avatar config has only the bitrate fields.

AvatarConfig::custom(image, "png") builds an avatar from a reference image instead. The API wants a PNG portrait of at least 704×1280, under 5 MB. Custom avatars and replicated_voice(..) (a voice cloned from a sample) are allow-listed features. You are responsible for the consents and rights needed to process a likeness or a voice.

Before this release, every inline part from the model was treated as audio. An avatar stream would have been played through the speaker as noise.

Extended thinking

ModelId::LIVE_3_8_EXTENDED_THINKING (gemini-3.8-live-extended-thinking, Google AI) thinks before answering, and requires a thinking level: a setup without one is refused (“Thinking level must be specified for this model”).

Live::builder()
    .model(ModelId::LIVE_3_8_EXTENDED_THINKING)
    .thinking_level(ThinkingLevel::Low);

A question that needs thought is answered in two turns. First comes a spoken holding line (“Let me calculate that for you.”), with SessionEvent::InteractionStatus("IN_PROGRESS"), and the turn completes. The answer then arrives on its own as the next turn, with no further input. Code that treats the first TurnComplete as “the answer is in” must wait for the next turn.

Transcription

Live::builder()
    .input_transcription_config(
        AudioTranscriptionConfig::default()
            .language_codes(["en-US", "es-US"])
            .custom_vocabulary(["QwikPay", "Xylotek"]),
    )
    .output_transcription();

Language hints (language_codes, BCP-47) reduce misdetected languages on short utterances. custom_vocabulary biases recognition toward product names, SKUs and proper nouns. Both are accepted on Google AI and Vertex AI.

Code execution

Gemini 3.8 Live, 3.8 Live Extended Thinking and 2.5 native audio on Google AI refuse the built-in codeExecution tool: the session closes at setup (1007, “Code Execution tool is not supported for this model”; measured 2026-10-11). The setup message leaves the tool off for these models and connect logs it among the ignored settings. To have the model compute over tool results, use code mode: a run_code tool whose programs call your own tools, each call governed like a direct one.

Tools: informative responses

Gemini 3.8 Live retries a tool with varied arguments when a response does not say why it came back empty. That can mean two to five sweeps before it speaks. Return a status the model can act on. Never return an empty object or null:

{
  "results": [],
  "status": "invalid_argument",
  "retryable": false,
  "message": "There is no provider named 'Mr. Johnson'. Ask the caller to confirm the name.",
  "valid_options": ["Dr. Johnson", "Dr. Chen"]
}

Pair this with a system-instruction retry policy, as in the example above.

A tool declared FunctionCallingBehavior::Blocking pauses the model until the response arrives. If the user speaks meanwhile, the server cancels the pending call. A response scheduled INTERRUPT while the user is speaking now waits until they finish.

Session lifecycle

• .history_in_client_content() sets historyConfig.initialHistoryInClientContent. Send the seed turns after setupComplete, with turnComplete: true on the last. With it, the model takes the turns in as history. Without it, it answers the seed as if the user had just spoken.
• .transparent_resumption() (Vertex AI) makes each resumption update name the last client message the server consumed (ResumeInfo::last_consumed_index). A resumed session then knows what to send again.
• .explicit_vad_signal() (Vertex AI) asks the server for voiceActivity events at the edges of user speech.

Audio formats

Unchanged: 16 kHz, 16-bit, little-endian mono PCM in (20–100 ms chunks), and 24 kHz mono PCM out.

References

• Developer’s guide to Gemini 3.8 Live
• Migrate from Gemini 2.5 Flash Live API to Gemini 3.8 Live
• Configure live avatars