Struct Live 

pub struct Live { /* private fields */ }

Fluent builder for constructing and connecting Gemini Live sessions.

Accumulates model configuration, callbacks, extractors, phases, watchers, temporal patterns, and tool execution modes, then connects via one of the connect_* methods.

Control-lane callbacks can be registered with _concurrent suffixed methods for fire-and-forget execution. Tools can be marked for background execution via tool_background().

Example

let session = Live::builder()
    .voice(Voice::Kore)
    .instruction("You are a weather assistant")
    .tools(tools)
    .on_audio(|data| { let _ = data; })
    .on_text(|t| print!("{t}"))
    .on_interrupted(|| async { /* flush playback */ })
    .connect_from_env()
    .await?;

Extraction Pipeline

let handle = Live::builder()
    .instruction("You are a restaurant order assistant")
    .extract_turns::<OrderState>(
        flash_llm,
        "Extract: items ordered, quantities, modifications, order_phase",
    )
    .on_extracted(|name, value| async move {
        println!("Extracted {name}: {value}");
    })
    .connect_from_env()
    .await?;

// Read latest extraction from shared State at any time:
let order: Option<OrderState> = handle.extracted("OrderState");

Implementations

impl Live

pub fn converse(self, convo: &CompiledConversation) -> Self

Drive a Live session from a compiled conversation: govern with its lowered flow and register the extractors that fill its frames’ slots each turn. The one-liner entrypoint for “run this conversation”.

let convo = Conversation::new("booking")
    .stage("done").terminal()
    .require(["done"])
    .compile()?;
let handle = Live::builder()
    .converse(&convo)
    .connect_from_env()
    .await?;

pub fn converse_observe(self, convo: &CompiledConversation) -> Self

Like converse but attaches the flow in observe mode (nothing blocked; deviations recorded) while still registering extractors.

impl Live

pub fn before_tool_response<F, Fut>(self, f: F) -> Self

where F: Fn(Vec<FunctionResponse>, State) -> Fut + Send + Sync + 'static, Fut: Future<Output = Vec<FunctionResponse>> + Send + 'static,

Intercept tool responses before they are sent back to Gemini.

Use this to rewrite, augment, or filter tool results based on conversation state. The callback receives the tool responses and the shared State, and returns (potentially modified) responses.

Example

Live::builder().before_tool_response(|responses, state| async move {
    let order: OrderState = state.get("OrderState").unwrap_or_default();
    responses.into_iter().map(|mut r| {
        r.response["current_order"] = serde_json::to_value(&order).unwrap();
        r
    }).collect()
});

pub fn on_turn_boundary<F, Fut>(self, f: F) -> Self

where F: Fn(State, Arc<dyn SessionWriter>) -> Fut + Send + Sync + 'static, Fut: Future<Output = ()> + Send + 'static,

Hook called at turn boundaries — after extractors run, before on_turn_complete.

Receives the shared State and a SessionWriter for injecting content into the conversation. Use for context stuffing, K/V data injection, condensed state summaries, or any outbound content interleaving.

Example

Live::builder().on_turn_boundary(|state, writer| async move {
    let summary = state.get::<String>("summary").unwrap_or_default();
    writer.send_client_content(
        vec![Content::user(format!("[Context: {summary}]"))],
        false,
    ).await.ok();
});

pub fn on_audio(self, f: impl Fn(&Bytes) + Send + Sync + 'static) -> Self

Called for each audio chunk from the model (PCM16 24kHz).

pub fn on_text(self, f: impl Fn(&str) + Send + Sync + 'static) -> Self

Called for each incremental text delta.

pub fn on_text_complete(self, f: impl Fn(&str) + Send + Sync + 'static) -> Self

Called when model completes a text response.

pub fn on_input_transcript( self, f: impl Fn(&str, bool) + Send + Sync + 'static, ) -> Self

Called for input (user speech) transcription: f(text, is_final).

While the user speaks, is_final is false and text is the latest partial recognition, which later calls may revise. At the turn boundary one call arrives with is_final == true carrying the complete transcript for the turn — the only value suitable for storage. Requires transcription or input_transcription.

pub fn on_output_transcript( self, f: impl Fn(&str, bool) + Send + Sync + 'static, ) -> Self

Called for output (model speech) transcription: f(text, is_final).

Same partial/final contract as on_input_transcript: is_final is false for revisable partials and true once for the turn’s complete transcript. Requires transcription or output_transcription.

pub fn on_thought(self, f: impl Fn(&str) + Send + Sync + 'static) -> Self

Called when the model emits a thought/reasoning summary.

Requires .include_thoughts() on the session config. Fast lane callback (sync, must complete in < 1ms).

pub fn on_vad_start(self, f: impl Fn() + Send + Sync + 'static) -> Self

Called when server VAD detects voice activity start.

pub fn on_vad_end(self, f: impl Fn() + Send + Sync + 'static) -> Self

Called when server VAD detects voice activity end.

pub fn on_usage( self, f: impl Fn(&UsageMetadata) + Send + Sync + 'static, ) -> Self

Called when server sends token usage metadata.

Receives a reference to the full UsageMetadata including prompt, response, cached, tool-use, and thoughts token counts plus per-modality breakdowns. Fires on the telemetry lane (not the fast lane).

pub fn on_session_phase( self, f: impl Fn(SessionPhase) + Send + Sync + 'static, ) -> Self

Called on wire-level session phase transitions (connecting → active → disconnecting …). This is the transport lifecycle, not the PhaseMachine (see .phase(..)).

Receives the new SessionPhase. Fast lane callback (sync, must complete in < 1ms). Use for lightweight UI state updates or metrics.

pub fn on_interrupted<F, Fut>(self, f: F) -> Self

where F: Fn() -> Fut + Send + Sync + 'static, Fut: Future<Output = ()> + Send + 'static,

Called when model is interrupted by barge-in.

Awaited before audio forwarding resumes, so a playback flush here is guaranteed to land before the next chunk.

pub fn on_tool_call<F, Fut>(self, f: F) -> Self

where F: Fn(Vec<FunctionCall>, State) -> Fut + Send + Sync + 'static, Fut: Future<Output = Option<Vec<FunctionResponse>>> + Send + 'static,

Called when model requests tool execution. Return None to auto-dispatch, Some(responses) to override. Receives State for natural state promotion from tool results.

pub fn on_tool_cancelled<F, Fut>(self, f: F) -> Self

where F: Fn(Vec<String>) -> Fut + Send + Sync + 'static, Fut: Future<Output = ()> + Send + 'static,

Called when the server cancels pending tool calls.

Receives the list of cancelled tool call IDs. Use to clean up any in-flight async work associated with those calls.

pub fn on_turn_complete<F, Fut>(self, f: F) -> Self

where F: Fn() -> Fut + Send + Sync + 'static, Fut: Future<Output = ()> + Send + 'static,

Called when model turn completes.

pub fn on_generation_complete<F, Fut>(self, f: F) -> Self

where F: Fn() -> Fut + Send + Sync + 'static, Fut: Future<Output = ()> + Send + 'static,

Called when the model finishes generating its full intended response.

Fires on the wire GenerationComplete event, before any interruption truncation. Use this to capture the model’s complete output even when the user barges in. Paired with .extract_on_generation() for structured extraction of the pre-truncation response.

pub fn on_go_away<F, Fut>(self, f: F) -> Self

where F: Fn(Duration) -> Fut + Send + Sync + 'static, Fut: Future<Output = ()> + Send + 'static,

Called when server sends GoAway.

pub fn on_connected<F, Fut>(self, f: F) -> Self

where F: Fn(Arc<dyn SessionWriter>) -> Fut + Send + Sync + 'static, Fut: Future<Output = ()> + Send + 'static,

Called when session connects (setup complete).

Receives a SessionWriter for sending messages on connect.

pub fn on_disconnected<F, Fut>(self, f: F) -> Self

where F: Fn(Option<String>) -> Fut + Send + Sync + 'static, Fut: Future<Output = ()> + Send + 'static,

Called when session disconnects.

pub fn on_teardown<F, Fut>(self, f: F) -> Self

where F: Fn() -> Fut + Send + Sync + 'static, Fut: Future<Output = ()> + Send + 'static,

Register an additive teardown hook, run on disconnect before on_disconnected.

Every other callback setter replaces: calling .on_disconnected(..) twice keeps only the second, silently. That is workable for an application and unusable for an extension, which cannot know whether the application will register a handler after it. Hooks registered here accumulate instead, so with_memory(..) and the application’s own on_disconnected both run regardless of the order they were written in.

Hooks are awaited in registration order before the session finishes tearing down, so this is the seam for flushing durable state. Keep them bounded — a hook that hangs delays disconnect.

Live::builder().on_teardown(|| async { /* flush */ });

pub fn on_resumed<F, Fut>(self, f: F) -> Self

where F: Fn() -> Fut + Send + Sync + 'static, Fut: Future<Output = ()> + Send + 'static,

Called after the session resumes following a GoAway disconnect.

Use to re-subscribe to external streams, reset UI state, or log resume events. Paired with .session_resume() on the builder.

pub fn on_error<F, Fut>(self, f: F) -> Self

where F: Fn(String) -> Fut + Send + Sync + 'static, Fut: Future<Output = ()> + Send + 'static,

Called on non-fatal errors with the error’s message.

The argument is a String, not a typed error: the runtime funnels server errors, codec failures, and processor faults into one human-readable message here, and the session keeps running. Fatal errors end the session and arrive through on_disconnected instead.

pub fn on_interrupted_concurrent<F, Fut>(self, f: F) -> Self

where F: Fn() -> Fut + Send + Sync + 'static, Fut: Future<Output = ()> + Send + 'static,

Called when model is interrupted by barge-in (spawned concurrently).

Audio forwarding resumes without waiting for the body, so this is for bookkeeping (metrics, a log line) — a playback flush must use the blocking on_interrupted.

pub fn on_turn_boundary_concurrent<F, Fut>(self, f: F) -> Self

where F: Fn(State, Arc<dyn SessionWriter>) -> Fut + Send + Sync + 'static, Fut: Future<Output = ()> + Send + 'static,

Turn-boundary hook spawned concurrently — for observation only. The next turn proceeds without waiting, so context injected from here is not guaranteed to precede it; use on_turn_boundary for that.

pub fn on_teardown_concurrent<F, Fut>(self, f: F) -> Self

where F: Fn() -> Fut + Send + Sync + 'static, Fut: Future<Output = ()> + Send + 'static,

An additive teardown hook spawned detached on disconnect rather than awaited — the disconnect does not wait for it. For a final metric or log line; anything that flushes durable state belongs in on_teardown.

pub fn on_turn_complete_concurrent<F, Fut>(self, f: F) -> Self

where F: Fn() -> Fut + Send + Sync + 'static, Fut: Future<Output = ()> + Send + 'static,

Called when model turn completes (spawned concurrently).

pub fn on_generation_complete_concurrent<F, Fut>(self, f: F) -> Self

where F: Fn() -> Fut + Send + Sync + 'static, Fut: Future<Output = ()> + Send + 'static,

Called when the model finishes generating its full intended response (spawned concurrently).

pub fn on_connected_concurrent<F, Fut>(self, f: F) -> Self

where F: Fn(Arc<dyn SessionWriter>) -> Fut + Send + Sync + 'static, Fut: Future<Output = ()> + Send + 'static,

Called when session connects (spawned concurrently).

pub fn on_disconnected_concurrent<F, Fut>(self, f: F) -> Self

where F: Fn(Option<String>) -> Fut + Send + Sync + 'static, Fut: Future<Output = ()> + Send + 'static,

Called when session disconnects (spawned concurrently).

pub fn on_resumed_concurrent<F, Fut>(self, f: F) -> Self

where F: Fn() -> Fut + Send + Sync + 'static, Fut: Future<Output = ()> + Send + 'static,

Called after session resumes from GoAway (spawned concurrently).

pub fn on_error_concurrent<F, Fut>(self, f: F) -> Self

where F: Fn(String) -> Fut + Send + Sync + 'static, Fut: Future<Output = ()> + Send + 'static,

Called on non-fatal errors (spawned concurrently).

pub fn on_go_away_concurrent<F, Fut>(self, f: F) -> Self

where F: Fn(Duration) -> Fut + Send + Sync + 'static, Fut: Future<Output = ()> + Send + 'static,

Called when server sends GoAway (spawned concurrently).

pub fn on_tool_cancelled_concurrent<F, Fut>(self, f: F) -> Self

where F: Fn(Vec<String>) -> Fut + Send + Sync + 'static, Fut: Future<Output = ()> + Send + 'static,

Called when the server cancels pending tool calls (spawned concurrently).

pub fn on_extracted_concurrent<F, Fut>(self, f: F) -> Self

where F: Fn(String, Value) -> Fut + Send + Sync + 'static, Fut: Future<Output = ()> + Send + 'static,

Called when a TurnExtractor produces a result (spawned concurrently).

pub fn on_extraction_error_concurrent<F, Fut>(self, f: F) -> Self

where F: Fn(String, String) -> Fut + Send + Sync + 'static, Fut: Future<Output = ()> + Send + 'static,

Called when a TurnExtractor fails (spawned concurrently).

impl Live

pub fn model(self, model: ModelId) -> Self

Set the Gemini model.

Without this, connect resolves a default the target platform actually serves (overridable via the GEMINI_MODEL environment variable) — see the connect_from_env docs on Live.

pub fn voice(self, voice: Voice) -> Self

Set the output voice.

pub fn instruction(self, instruction: impl Into<String>) -> Self

Set the system instruction.

pub fn text_only(self) -> Self

Switch to text-only mode (no audio output).

Sets response modality to Text and disables speech config. Use with ModelId::LIVE_2_5_FLASH_NATIVE_AUDIO for text-only conversations.

pub fn add_tool(self, tool: Tool) -> Self

Add a raw Tool declaration to the session configuration.

Use this for tools that aren’t registered through the ToolDispatcher (e.g., raw FunctionDeclaration lists, Google Search, code execution).

pub fn greeting(self, prompt: impl Into<String>) -> Self

Set a greeting prompt to trigger the model to initiate the conversation.

When set, this text is sent immediately after the session connects, causing the model to respond first (e.g. with a greeting or introduction).

let handle = Live::builder()
    .instruction("You are a friendly assistant")
    .greeting("Greet the user warmly and introduce yourself.")
    .connect_from_env()
    .await?;
// Model will speak first without any user input

pub fn temperature(self, temp: f32) -> Self

Set the temperature.

pub fn record_wire(self, path: impl Into<PathBuf>) -> Self

Record every wire byte (both directions) to a JSONL log at path.

The log is written by a FileWireRecorder created at connect time (a connect error is returned if the file cannot be created). Replay it offline with adk session replay <path> or gemini_adk_rs::live::replay::replay_session.

let handle = Live::builder()
    .record_wire("/tmp/session.wire.jsonl")
    .connect_from_env()
    .await?;

pub fn wire_recorder(self, recorder: Arc<dyn WireRecorder>) -> Self

Record every wire byte to a custom WireRecorder implementation.

Overrides (and is overridden by) the most recent of this and record_wire.

pub fn tools(self, tools: impl Into<ToolComposite>) -> Self

Register tools: a |-composed ToolComposite from the T namespace, or a single ToolFunction (a SimpleTool/TypedTool, the value a #[tool] function returns, an Arc<dyn ToolFunction>).

Runtime tools go into the session’s dispatcher (created on demand), built-ins into the session config, agent tools share the session State, and T::mcp(..) toolsets are connected at connect.

Live::builder()
    .tools(
        T::simple("get_weather", "Get weather", |args| async move {
            let _ = args;
            Ok(serde_json::json!({"temp": 22}))
        })
        | T::google_search()
    );

pub fn tool(self, f: impl ToolFunction + 'static) -> Self

Register one tool — anything that implements ToolFunction.

#[tool("Get the weather for a city")]
async fn get_weather(city: String) -> Result<serde_json::Value, ToolError> {
    Ok(serde_json::json!({"city": city, "temp": 22}))
}
Live::builder().tool(get_weather());

pub fn dispatcher(self, dispatcher: ToolDispatcher) -> Self

Use a ToolDispatcher you built yourself as the session’s dispatcher — the escape hatch for streaming tools, input-streaming tools, or a dispatcher shared with other components. Replaces any dispatcher the builder created for tools so far; tools registered after this call are added to it.

pub fn agent_tool( self, name: impl Into<String>, description: impl Into<String>, agent: impl TextAgent + 'static, ) -> Self

Register a text agent as a tool the live model can call.

The agent shares the session’s State, so it can read live-extracted values and its mutations are visible to watchers and phase transitions.

Live::builder()
    .agent_tool("verify_identity", "Verify caller identity", verifier_agent)
    .agent_tool("calc_payment", "Calculate payment plans", calc_pipeline);

pub fn agent_tool_arc( self, name: impl Into<String>, description: impl Into<String>, agent: Arc<dyn TextAgent>, ) -> Self

Register a text agent (already Arc’d) as a tool.

pub fn google_search(self) -> Self

Enable Google Search built-in tool.

pub fn code_execution(self) -> Self

Enable code execution built-in tool.

pub fn url_context(self) -> Self

Enable URL context built-in tool.

pub fn tool_background(self, tool_name: impl Into<String>) -> Self

Mark a tool for background execution (zero dead-air).

When the model calls this tool, an immediate “running” acknowledgment is sent back while the tool executes in a background task. The final result is delivered asynchronously when complete.

pub fn tool_background_with_formatter( self, tool_name: impl Into<String>, formatter: Arc<dyn ResultFormatter>, ) -> Self

Mark a tool for background execution with a custom result formatter.

The formatter controls the shape of the acknowledgment (“running”), completion, and cancellation messages sent to the model.

pub fn tool_background_with_scheduling( self, tool_name: impl Into<String>, scheduling: FunctionResponseScheduling, ) -> Self

Mark a tool for background execution with a specific scheduling mode.

The scheduling mode controls how the model handles async results:

• Interrupt: halts current output, immediately reports the result
• WhenIdle: waits until current output finishes before handling
• Silent: integrates the result without notifying the user

pub fn transcription(self) -> Self

Enable transcription of both the user’s speech and the model’s audio (delivered to on_input_transcript / on_output_transcript).

pub fn input_transcription(self) -> Self

Enable transcription of the user’s speech only.

pub fn output_transcription(self) -> Self

Enable transcription of the model’s audio output only.

pub fn thinking(self, budget: u32) -> Self

Enable thinking/reasoning with a token budget (Gemini 2.5+).

Sets the thinking budget for the Live session. Use with .include_thoughts() and .on_thought() to receive thought summaries.

Live::builder()
    .thinking(1024)
    .include_thoughts()
    .on_thought(|text| println!("[Thought] {text}"));

Platform support: Google AI only. On Vertex AI, thinkingConfig is automatically stripped from the setup message.

pub fn include_thoughts(self) -> Self

Include the model’s thought summaries in responses.

When enabled, the model emits SessionEvent::Thought events containing its reasoning process. Register an .on_thought() callback to receive them.

Platform support: Google AI only. Stripped on Vertex AI.

pub fn affective_dialog(self) -> Self

Enable affective dialog (emotionally expressive responses).

pub fn proactive_audio(self) -> Self

Enable proactive audio: the model may decide not to respond to input it judges not addressed to it. Pair with soft_turn_timeout.

pub fn media_resolution(self, res: MediaResolution) -> Self

Set media resolution for video/image input.

pub fn mic_denoise(self) -> Self

Run the RNNoise speech enhancer over outgoing mic audio inside send_audio (feature denoise) — the same stage as voice::Denoiser, applied server-side to hosted surfaces (web bridge, API server) that do not run a local pump. See the hardening chapter for the measured benchmark.

pub fn mic_noise_gate(self, threshold_rms: f64, hold_frames: u32) -> Self

Chain a NoiseGate over outgoing mic audio inside send_audio, after any denoiser — frames whose RMS falls below threshold_rms are silenced, with hold_frames of hangover. Calibrate the threshold between the caller’s level and the background (measured sweet spot 400–700 behind the denoiser).

pub fn mic_processor( self, processor: impl InputAudioProcessor + 'static, ) -> Self

Chain any InputAudioProcessor over outgoing mic audio inside send_audio — the open slot for application-side stages (a DeepFilterNet enhancer, AGC, a custom filter). Stages run in the order configured.

pub fn input_vad(self, config: VadConfig) -> Self

Replace the input VAD’s configuration (the detector that runs inside send_audio for client-side speech edges). Use VadConfig::noisy_street() behind mic_denoise for noisy environments.

pub fn client_interruption_authority(self) -> Self

Give this client’s input VAD interruption authority: the session is configured with the server’s automatic activity detection disabled, and send_audio emits activityStart/activityEnd on the input VAD’s speech edges. Measured ~2× faster barge-in than server authority; pair with mic_denoise (and mic_noise_gate) or noise will drive the marks.

pub fn turn_commit(self, config: TurnCommitConfig) -> Self

Install a turn-commit policy between the input VAD’s speech edges and the activity marks sent under client_interruption_authority.

Raw edges make two measured mistakes as turn signals (TurnBench dev set, 38 real dyadic conversations): committing end-of-turn during mid-turn pauses (fp 0.206 raw -> 0.087 with an 800 ms hold), and treating backchannels (“mm-hm”) over model speech as barge-ins (fp 0.702 raw -> 0.062 with a 1400 ms sustain). See TurnCommitConfig for the presets carrying those operating points.

pub fn turn_commit_ms(self, eot_hold_ms: u64, min_interruption_ms: u64) -> Self

turn_commit with millisecond knobs.

pub fn turn_commit_eot_hold_ms(self, eot_hold_ms: u64) -> Self

Set only the end-of-turn hold, keeping (or defaulting) the sustain — one of the two granular forms Flow Studio codegen emits.

pub fn turn_commit_min_interruption_ms(self, min_interruption_ms: u64) -> Self

Set only the interruption sustain, keeping (or defaulting) the hold.

pub fn vad(self, detection: AutomaticActivityDetection) -> Self

Configure server-side VAD.

pub fn activity_handling(self, handling: ActivityHandling) -> Self

Set activity handling mode (interrupts vs no-interruption).

pub fn turn_coverage(self, coverage: TurnCoverage) -> Self

Set turn coverage mode.

pub fn session_resume(self) -> Self

Enable session resumption: the server issues resumption handles that session_resume_from accepts on the next connect.

pub fn session_resume_from(self, handle: impl Into<String>) -> Self

Resume a previous session from a server-issued resumption handle.

Capture the handle from the old session before it ends — via LiveHandle::resume_handle (e.g. inside the on_go_away callback) or from a persisted SessionSnapshot — and pass it here on the next connect. Resumption stays enabled for the new session, so fresh handles keep arriving. No automatic reconnect is performed.

pub fn context_compression( self, trigger_tokens: u32, target_tokens: u32, ) -> Self

Set the context window compression thresholds.

Compression is on by default at DEFAULT_COMPRESSION_TRIGGER_TOKENS / DEFAULT_COMPRESSION_TARGET_TOKENS, so a long call keeps going instead of ending when the model’s context fills. Use this to tune the thresholds; no_context_compression turns it off.

pub fn no_context_compression(self) -> Self

Disable context window compression.

Without it the server ends the session once the model’s context window is full. Turn it off only when a session is known to be short, or when the whole history must stay verbatim for its full length.

pub fn soft_turn_timeout(self, timeout: Duration) -> Self

Enable soft turn detection for proactive silence awareness.

When proactiveAudio is enabled, the model may choose not to respond. After VAD end, if the model stays silent for timeout, a lightweight “soft turn” updates state and fires watchers without forcing a response.

pub fn steering_mode(self, mode: SteeringMode) -> Self

Set the steering mode for how the phase machine delivers instructions.

• InstructionUpdate (default): Replace system instruction on transition.
• ContextInjection: Inject steering via send_client_content.
• Hybrid: Instruction on transition, context injection per turn.

pub fn context_delivery(self, mode: ContextDelivery) -> Self

Set when model-role context turns are delivered to the wire.

• Immediate (default): Send as a single batched frame during TurnComplete processing.
• Deferred: Queue context and flush before the next user send (send_audio/send_text/send_video). Eliminates isolated WebSocket frames during silence that can confuse the model.

Live::builder()
    .steering_mode(SteeringMode::ContextInjection)
    .context_delivery(ContextDelivery::Deferred)
    .phase("greeting")
        .instruction("Welcome the guest")
        .done()
    .initial_phase("greeting");

pub fn delivery(self, delivery: DeliveryConfig) -> Self

Set the fast-lane delivery (backpressure) policy for every event class.

The event router forwards fast-lane frames (audio, text, transcripts, thoughts, VAD, phase) to the fast-lane consumer over a bounded channel. By default every class is Delivery::Lossless — the router awaits (send().await) when the channel is full, which preserves the historical behavior. Opt classes into Delivery::LossyDropNewest to drop the newest frame on overflow instead of stalling the router (and thereby stalling control-lane routing too).

use gemini_adk_fluent_rs::live::{Delivery, DeliveryConfig};
Live::builder()
    .delivery(DeliveryConfig::default()
        .audio(Delivery::LossyDropNewest)
        .transcript(Delivery::LossyDropNewest));

pub fn lossy_audio(self) -> Self

Convenience: set the audio class to Delivery::LossyDropNewest so the router never blocks on a slow audio consumer, dropping the newest PCM frame on overflow. Other classes keep their current policy.

pub fn lossy_transcript(self) -> Self

Convenience: set the transcript class to Delivery::LossyDropNewest. Other classes keep their current policy.

pub fn redaction(self, redactor: TranscriptRedactor) -> Self

Install transcript redaction: sensitive strings (card numbers, one-time codes, custom patterns) are removed at the event router, before callbacks, the transcript buffer, extraction, or persistence see the text.

use gemini_adk_rs::live::redaction::TranscriptRedactor;
Live::builder()
    .redaction(TranscriptRedactor::new().card_numbers().long_digits(6));

pub fn repair(self, config: RepairConfig) -> Self

Enable the conversation repair protocol.

Tracks unfulfilled needs per phase. After nudge_after stalled turns, injects a gentle nudge. After escalate_after turns, sets repair:escalation in state for phase guards to handle.

pub fn persistence(self, backend: Arc<dyn SessionPersistence>) -> Self

Set a session persistence backend for surviving process restarts.

pub fn session_id(self, id: impl Into<String>) -> Self

Set the session ID for persistence.

pub fn no_tool_advisory(self) -> Self

Disable the tool availability advisory on phase transitions.

By default the SDK injects a model-role context turn telling the model which tools are available in the new phase.

impl Live

pub async fn connect_google_ai( self, api_key: impl Into<String>, ) -> Result<LiveHandle, AgentError>

Connect using a Google AI API key.

pub async fn connect_vertex( self, project: impl Into<String>, location: impl Into<String>, access_token: impl Into<AccessToken>, ) -> Result<LiveHandle, AgentError>

Connect using Vertex AI credentials.

access_token is a fixed token or an AccessToken; pass AccessToken::from_fn(..) for a source that is consulted on every reconnect, since Vertex tokens expire after about an hour.

pub async fn connect_from_env(self) -> Result<LiveHandle, AgentError>

Connect by resolving the platform and credentials from standard environment variables — the zero-ceremony entry point.

Resolution (see ApiEndpoint::from_env):

• GOOGLE_GENAI_USE_VERTEXAI=true → Vertex AI using GOOGLE_CLOUD_PROJECT, GOOGLE_CLOUD_LOCATION (default us-central1), and a token from GOOGLE_ACCESS_TOKEN. If that token is unset, this falls back to running gcloud auth print-access-token.
• otherwise → Google AI using GEMINI_API_KEY (or GOOGLE_GENAI_API_KEY / GOOGLE_API_KEY).

When no .model(..) was set, the wire layer resolves a default the target platform actually serves (ModelId::live_default): GEMINI_MODEL from the environment if present, else the platform’s current native-audio Flash model.

let handle = Live::builder()
    .voice(Voice::Kore)
    .connect_from_env()
    .await?;

pub async fn connect( self, config: SessionConfig, ) -> Result<LiveHandle, AgentError>

Connect using a pre-configured SessionConfig for auth and model.

Merges the provided config’s endpoint and model into the builder’s config, preserving system instruction, tools, voice, transcription, and all other settings configured via the fluent API.

impl Live

pub fn describe_contract(&self) -> RuntimeContract

Describe the configured runtime contract before the session connects.

The contract is intended for DevTools, replay validation, and generated docs. It is metadata only; predicates and callbacks are represented by stable names or boolean capabilities rather than executable closures.

impl Live

pub fn extract_turns<T>( self, llm: Arc<dyn BaseLlm>, prompt: impl Into<String>, ) -> Self

where T: DeserializeOwned + Serialize + JsonSchema + Send + Sync + 'static,

Add a turn extractor that runs an OOB LLM after each turn to extract structured data from the transcript window.

Automatically enables both input and output transcription. The extraction result is stored in State under the type name (e.g., "OrderState") and can be read via handle.extracted::<T>(name).

The type T must implement JsonSchema for schema-guided extraction. The window size defaults to 3 turns.

pub fn extract_record(self, spec: Extract) -> Self

Register a deterministic Extract record — CPU recognizers over the transcript, no model, no network. The recognized fields are promoted into State, where Flow guards (done(captured([...]))) and repair read them. Composes with extract_turns (LLM) on the same session for a cheap-first cascade.

pub fn extract_turns_windowed<T>( self, llm: Arc<dyn BaseLlm>, prompt: impl Into<String>, window_size: usize, ) -> Self

where T: DeserializeOwned + Serialize + JsonSchema + Send + Sync + 'static,

Like extract_turns but with a custom window size.

pub fn extract_turns_triggered<T>( self, llm: Arc<dyn BaseLlm>, prompt: impl Into<String>, window_size: usize, trigger: ExtractionTrigger, ) -> Self

where T: DeserializeOwned + Serialize + JsonSchema + Send + Sync + 'static,

Like extract_turns_windowed but with a custom extraction trigger.

Use ExtractionTrigger::AfterToolCall when tool calls are the primary state source, ExtractionTrigger::Interval(n) to reduce extraction frequency, or ExtractionTrigger::OnPhaseChange for phase-entry extraction.

pub fn extract_turns_configured<T>( self, llm: Arc<dyn BaseLlm>, prompt: impl Into<String>, window_size: usize, trigger: ExtractionTrigger, configure: impl FnOnce(LlmExtractor) -> LlmExtractor, ) -> Self

where T: DeserializeOwned + Serialize + JsonSchema + Send + Sync + 'static,

Like extract_turns_triggered, but lets callers configure the underlying LlmExtractor before registration.

Use this for field promotion rules, custom minimum word counts, or other extraction policies that should live at the SDK layer instead of app callback glue.

pub fn extract_json( self, llm: Arc<dyn BaseLlm>, name: impl Into<String>, schema: Value, prompt: impl Into<String>, ) -> Self

Like extract_turns, but schema-as-data: no Rust type required. The extraction result is stored in State under name; pair with FieldPromotion rules via extractor (or a SessionSpec extract entry, which wires promotions declaratively) to land individual fields in the bare keys flow guards read.

This is the piece that lets a JSON-authored flow advance from speech alone: extraction fills the state that captured/is_true guards latch on, with no tool call anywhere.

pub fn extractor(self, extractor: Arc<dyn TurnExtractor>) -> Self

Add a custom TurnExtractor implementation.

pub fn on_extracted<F, Fut>(self, f: F) -> Self

where F: Fn(String, Value) -> Fut + Send + Sync + 'static, Fut: Future<Output = ()> + Send + 'static,

Called when a TurnExtractor produces a result.

The callback receives the extractor name and the extracted JSON value.

pub fn on_extraction_error<F, Fut>(self, f: F) -> Self

where F: Fn(String, String) -> Fut + Send + Sync + 'static, Fut: Future<Output = ()> + Send + 'static,

Called when a TurnExtractor fails.

The callback receives the extractor name and error message. Use this for custom error handling (alerting, retry logic, etc.).

impl Live

pub fn declared_tool_names(&self) -> Vec<String>

Every tool name this builder can name without connecting.

Covers config-level declarations (google_search, code_execution, …), everything registered on the dispatcher, and deferred agent tools, whose names are known up front even though the tool is built at connect.

Does not cover MCP/A2A/OpenAPI tools, whose names live on the far side of an async handshake — see pending_tool_count. Duplicates are removed; order is not meaningful.

pub fn pending_tool_count(&self) -> usize

How many tools are still unresolved, and so absent from declared_tool_names.

Non-zero means any name-based check run now is working from a partial picture. Zero means declared_tool_names is the whole set.

pub fn flow(&self) -> Option<&Flow>

The governing flow, if govern or observe was called.

The flow’s own ambient list is as the caller wrote it; tools registered through ambient_tools are merged in at connect and are readable separately via ambient_tool_names.

pub fn flow_enforcement(&self) -> Enforcement

Whether an attached flow is enforced or merely observed.

pub fn phases(&self) -> &[Phase]

The configured phases, in declaration order.

pub fn initial_phase_name(&self) -> Option<&str>

The initial phase name, if one was set.

A phase machine with phases but no initial phase never starts, which is why this is worth being able to ask about.

pub fn watched_keys(&self) -> Vec<String>

The state keys under watch, via watch.

pub fn extractor_count(&self) -> usize

How many turn extractors are installed.

Extractors are opaque trait objects — this reports presence, which is enough to tell a session that will populate state from one that will not.

pub fn has_persistence(&self) -> bool

Whether a session persistence backend is attached.

pub fn shared_state(&self) -> Option<&State>

The caller-supplied session State, if state was called.

pub fn teardown_hook_count(&self) -> usize

How many additive teardown hooks are registered, via on_teardown.

Lets an extension assert its own end-of-session wiring: with_memory installs one here, and a session that reports zero will not persist anything it learned.

impl Live

pub fn instruction_template( self, f: impl Fn(&State) -> Option<String> + Send + Sync + 'static, ) -> Self

State-reactive system instruction template.

Called after extractors run on each turn. If it returns Some(instruction), the system instruction is updated mid-session (deduped — same instruction is not sent twice). Returns None to leave the instruction unchanged.

Example

Live::builder().instruction_template(|state| {
    let phase: String = state.get("phase").unwrap_or_default();
    match phase.as_str() {
        "ordering" => Some("Focus on taking the order accurately.".into()),
        "confirming" => Some("Summarize and confirm the order.".into()),
        _ => None,
    }
});

pub fn instruction_amendment( self, f: impl Fn(&State) -> Option<String> + Send + Sync + 'static, ) -> Self

State-reactive instruction amendment (additive, not replacement).

Unlike instruction_template (which replaces the entire instruction), this appends to the current phase instruction. The developer never needs to know or repeat the base instruction.

Example

Live::builder().instruction_amendment(|state| {
    let risk: String = state.get("derived:risk").unwrap_or_default();
    if risk == "high" {
        Some("[IMPORTANT: Use empathetic language. Do not threaten.]".into())
    } else {
        None
    }
});

pub fn computed( self, key: impl Into<String>, deps: &[&str], f: impl Fn(&State) -> Option<Value> + Send + Sync + 'static, ) -> Self

Register a computed (derived) state variable.

The compute function receives the full State and returns Some(value) to write to derived:{key}, or None to skip.

A dependency cycle among computed variables is a configuration error; it is reported by connect (as AgentError::Config), never a panic.

pub fn phase_defaults( self, f: impl FnOnce(PhaseDefaults) -> PhaseDefaults, ) -> Self

Set default modifiers and prompt_on_enter inherited by all phases.

Phase-specific modifiers are applied after defaults, so they extend (not replace).

Live::builder()
    .phase_defaults(|p| {
        p.show_state(&["emotional_state", "risk_level"])
         .when(|s| s.get::<String>("risk").unwrap_or_default() == "high", "Show extra empathy.")
         .prompt_on_enter()
    })
    .phase("greet").instruction("...").done()
    .phase("close").instruction("...").done();
    // Both phases inherit the modifiers and prompt_on_enter.

pub fn phase(self, name: impl Into<String>) -> PhaseBuilder

Start building a conversation phase.

Returns a PhaseBuilder that flows back to this Live via .done().

pub fn initial_phase(self, name: impl Into<String>) -> Self

Set the initial phase name (must match a registered phase).

pub fn watch(self, key: impl Into<String>) -> WatchBuilder

Start building a state watcher.

Returns a WatchBuilder that flows back to this Live via .then().

pub fn when_sustained<F, Fut>( self, name: impl Into<String>, condition: impl Fn(&State) -> bool + Send + Sync + 'static, duration: Duration, action: F, ) -> Self

where F: Fn(State, Arc<dyn SessionWriter>) -> Fut + Send + Sync + 'static, Fut: Future<Output = ()> + Send + 'static,

Register a sustained condition pattern.

Fires when the condition remains true for at least duration.

pub fn when_rate<F, Fut>( self, name: impl Into<String>, filter: impl Fn(&SessionEvent) -> bool + Send + Sync + 'static, count: u32, window: Duration, action: F, ) -> Self

where F: Fn(State, Arc<dyn SessionWriter>) -> Fut + Send + Sync + 'static, Fut: Future<Output = ()> + Send + 'static,

Register a rate detection pattern.

Fires when at least count matching events occur within window.

pub fn when_turns<F, Fut>( self, name: impl Into<String>, condition: impl Fn(&State) -> bool + Send + Sync + 'static, turn_count: u32, action: F, ) -> Self

where F: Fn(State, Arc<dyn SessionWriter>) -> Fut + Send + Sync + 'static, Fut: Future<Output = ()> + Send + 'static,

Register a turn count pattern.

Fires when the condition is true for turn_count consecutive turns.

impl Live

pub fn builder() -> Self

Start building a Live session.

Examples

Minimal live session setup:

let handle = Live::builder()
    .voice(Voice::Kore)
    .instruction("You are a helpful assistant")
    .greeting("Hello! How can I help?")
    .on_audio(|data| { let _ = data; /* send to speaker */ })
    .on_text(|t| print!("{t}"))
    .connect_google_ai("API_KEY")
    .await?;

handle.send_text("What is the weather?").await?;
handle.disconnect().await?;

With phases and state-based transitions:

let handle = Live::builder()
    .phase("greeting")
        .instruction("Welcome the user")
        .transition("main", S::is_true("greeted"))
        .done()
    .phase("main")
        .instruction("Help the user")
        .terminal()
        .done()
    .initial_phase("greeting")
    .connect_google_ai("API_KEY")
    .await?;

pub fn govern(self, flow: Flow) -> Self

Govern the session with a Flow DAG and enforce it: inadmissible tool calls are blocked and active-step postures steer the model at each turn boundary.

pub fn state(self, state: State) -> Self

Use a State you already hold as the session’s state.

Without this a tool closure captures whatever State the caller made, the session runs on a different one, and the two never meet — so a tool that writes identity_verified and a Guard::is_true("identity_verified") that reads it are talking about different maps. The guard never fires, the flow never advances, and every subsequent tool is refused by a gate whose condition was in fact satisfied.

That is the ordinary shape of a governed flow — tools write the facts, guards read them — so this is how you make it work:

let state = State::new();
Live::builder()
    .state(state.clone())   // the session runs on this
    .tools(my_tools(state)); // and so do the tools

agent_tool already shares state with the agents it wraps; this is the same guarantee for ordinary tools.

pub fn ambient_tools<I, S>(self, tools: I) -> Self

where I: IntoIterator<Item = S>, S: Into<String>,

Register cross-cutting tools as ambient: exempt from every step’s allow whitelist, still bound by anything that names them.

Merged into the governing flow at connect, so this composes with govern in either order. Without a flow it is inert.

Extensions that install their own tools should call this rather than making the application remember to widen every step — with_memory does exactly that for recall_context and manage_memory.

pub fn ambient_tool_names(&self) -> &[String]

The cross-cutting tools registered via ambient_tools.

Introspection for extensions and tests: the flow’s own ambient list is not included, because the two are only merged at connect.

pub fn observe(self, flow: Flow) -> Self

Attach a Flow in observe mode: nothing is blocked, but deviations are recorded for audit/analytics.

pub fn govern_compiled(self, flow: CompiledFlow) -> Self

Govern the session with a pre-compiled CompiledFlow and enforce it.

A CompiledFlow carries proof that Flow::compile (or Flow::compile_with_tools) already surfaced its diagnostics, so connect does not re-validate or re-compile it — compile once at load time, govern many sessions.

pub fn observe_compiled(self, flow: CompiledFlow) -> Self

Attach a pre-compiled CompiledFlow in observe mode: nothing is blocked, but deviations are recorded for audit/analytics. Like govern_compiled, the flow is not re-validated or re-compiled at connect.

pub fn on_step_enter( self, step: impl Into<String>, agent: Arc<dyn TextAgent>, mode: AgentMode, ) -> Self

Run an agent the first time the named flow step becomes active.

The agent reads its inputs from State and its result lands in {step}:result (AgentMode::Call resolves inline at the turn boundary; AgentMode::Dispatch/AgentMode::Background run detached). A downstream step can then complete on it via Guard::resolved(step). This is how a governed flow drives in-session orchestration. Requires a flow (govern/observe).

pub fn confirmation_provider( self, provider: Arc<dyn ConfirmationProvider>, ) -> Self

Gate T::confirm(..) tools behind a confirmation provider.

When set, any confirmation-gated tool is checked against provider before it runs; a denied decision returns an error to the model instead of executing the tool. Accepts any ConfirmationProvider — including a plain async closure of Fn(ConfirmationRequest) -> impl Future<Output = ToolConfirmation>.

pub fn middleware(self, middleware: impl Into<MiddlewareComposite>) -> Self

Attach middleware — a MiddlewareComposite or a single Arc<dyn Middleware> — every layer runs around tool dispatch in the control lane (before_tool can veto a call, after_tool and on_tool_error observe results).

Compose layers with |, e.g. M::log() | M::latency().

Note: model-level hooks (before_model/after_model) are TextAgent pipeline concepts and do not apply to a streaming Live session.

pub fn telemetry_interval(self, interval: Duration) -> Self

Set the periodic telemetry emission interval.

When set, the processor emits LiveEvent::Telemetry snapshots and LiveEvent::TurnMetrics at this rate.