Trait BaseLlm 

pub trait BaseLlm: Send + Sync {
    // Required methods
    fn model_id(&self) -> &str;
    fn generate<'life0, 'async_trait>(
        &'life0 self,
        request: LlmRequest,
    ) -> Pin<Box<dyn Future<Output = Result<LlmResponse, LlmError>> + Send + 'async_trait>>
       where Self: 'async_trait,
             'life0: 'async_trait;

    // Provided methods
    fn capabilities(&self) -> ModelCapabilities { ... }
    fn generate_stream<'life0, 'async_trait>(
        &'life0 self,
        request: LlmRequest,
    ) -> Pin<Box<dyn Future<Output = Result<LlmStream, LlmError>> + Send + 'async_trait>>
       where Self: 'async_trait,
             'life0: 'async_trait { ... }
    fn warm_up<'life0, 'async_trait>(
        &'life0 self,
    ) -> Pin<Box<dyn Future<Output = Result<(), LlmError>> + Send + 'async_trait>>
       where Self: 'async_trait,
             'life0: 'async_trait { ... }
}

Trait for LLM providers — decouples agents from specific models.

Implementations must be Send + Sync for use across async tasks.

Required Methods

fn model_id(&self) -> &str

The model identifier (e.g., “gemini-2.5-flash”).

fn generate<'life0, 'async_trait>( &'life0 self, request: LlmRequest, ) -> Pin<Box<dyn Future<Output = Result<LlmResponse, LlmError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Generate content from the LLM.

Provided Methods

fn capabilities(&self) -> ModelCapabilities

What this model supports — the ADK model-capability-declaration pattern: the model states its capabilities instead of every caller re-inferring them from the id string. The default derives a conservative estimate from model_id; back-ends with authoritative knowledge should override.

fn generate_stream<'life0, 'async_trait>( &'life0 self, request: LlmRequest, ) -> Pin<Box<dyn Future<Output = Result<LlmStream, LlmError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Generate content as a stream of chunks, in the order the model produced them. LlmResponse::append folds them into the whole reply; usage, when a chunk carries it, is cumulative.

The default yields generate’s reply as one chunk, so every provider supports it; GeminiLlm streams for real.

fn warm_up<'life0, 'async_trait>( &'life0 self, ) -> Pin<Box<dyn Future<Output = Result<(), LlmError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Pre-warm the HTTP connection pool to avoid cold-start latency.

The default implementation is a no-op. GeminiLlm overrides this to establish the TCP+TLS connection so the first real generate() call doesn’t pay the ~100-300ms handshake penalty.

Implementations on Foreign Types

impl<L: BaseLlm + ?Sized> BaseLlm for Box<L>

A boxed model is a model; see the Arc implementation.

fn model_id(&self) -> &str

fn capabilities(&self) -> ModelCapabilities

fn generate<'life0, 'async_trait>( &'life0 self, request: LlmRequest, ) -> Pin<Box<dyn Future<Output = Result<LlmResponse, LlmError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

fn generate_stream<'life0, 'async_trait>( &'life0 self, request: LlmRequest, ) -> Pin<Box<dyn Future<Output = Result<LlmStream, LlmError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

fn warm_up<'life0, 'async_trait>( &'life0 self, ) -> Pin<Box<dyn Future<Output = Result<(), LlmError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

impl<L: BaseLlm + ?Sized> BaseLlm for Arc<L>

A shared model is a model, so Arc<GeminiLlm>, Arc<dyn BaseLlm> and a bare GeminiLlm are interchangeable wherever a BaseLlm is accepted.

fn model_id(&self) -> &str

fn capabilities(&self) -> ModelCapabilities

fn generate<'life0, 'async_trait>( &'life0 self, request: LlmRequest, ) -> Pin<Box<dyn Future<Output = Result<LlmResponse, LlmError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

fn generate_stream<'life0, 'async_trait>( &'life0 self, request: LlmRequest, ) -> Pin<Box<dyn Future<Output = Result<LlmStream, LlmError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

fn warm_up<'life0, 'async_trait>( &'life0 self, ) -> Pin<Box<dyn Future<Output = Result<(), LlmError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Implementors

impl BaseLlm for GeminiLlm

impl BaseLlm for MockLlm

impl<L: BaseLlm> BaseLlm for TapedLlm<L>