Struct CacheMiddleware 

pub struct CacheMiddleware { /* private fields */ }

Caches model responses keyed by request hash to avoid redundant LLM calls.

Implementations

impl CacheMiddleware

pub fn len(&self) -> usize

Returns the number of cached entries.

pub fn is_empty(&self) -> bool

Whether the cache is empty.

pub fn clear(&self)

Clear all cached entries.

Trait Implementations

impl Middleware for CacheMiddleware

fn name(&self) -> &str

Unique name for this middleware (used in logging/debugging).

fn before_model<'life0, 'life1, 'async_trait>( &'life0 self, request: &'life1 LlmRequest, ) -> Pin<Box<dyn Future<Output = Result<Option<LlmResponse>, AgentError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Called before an LLM model call is made. Return Some(LlmResponse) to skip the LLM call and use the returned response instead (e.g., for caching, guardrails). Return None to proceed.

fn after_model<'life0, 'life1, 'life2, 'async_trait>( &'life0 self, request: &'life1 LlmRequest, response: &'life2 LlmResponse, ) -> Pin<Box<dyn Future<Output = Result<Option<LlmResponse>, AgentError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait, 'life2: 'async_trait,

Called after an LLM model call completes. Return Some(LlmResponse) to replace the model’s response (e.g., for output filtering, safety). Return None to use the original response.

fn before_agent<'life0, 'life1, 'async_trait>( &'life0 self, _ctx: &'life1 InvocationContext, ) -> Pin<Box<dyn Future<Output = Result<(), AgentError>> + Send + 'async_trait>>

where 'life0: 'async_trait, 'life1: 'async_trait, Self: 'async_trait,

Called before an agent begins execution.

fn after_agent<'life0, 'life1, 'async_trait>( &'life0 self, _ctx: &'life1 InvocationContext, ) -> Pin<Box<dyn Future<Output = Result<(), AgentError>> + Send + 'async_trait>>

where 'life0: 'async_trait, 'life1: 'async_trait, Self: 'async_trait,

Called after an agent completes execution.

fn before_tool<'life0, 'life1, 'async_trait>( &'life0 self, _call: &'life1 FunctionCall, ) -> Pin<Box<dyn Future<Output = Result<(), AgentError>> + Send + 'async_trait>>

where 'life0: 'async_trait, 'life1: 'async_trait, Self: 'async_trait,

Called before a tool is invoked.

fn after_tool<'life0, 'life1, 'life2, 'async_trait>( &'life0 self, _call: &'life1 FunctionCall, _result: &'life2 Value, ) -> Pin<Box<dyn Future<Output = Result<(), AgentError>> + Send + 'async_trait>>

where 'life0: 'async_trait, 'life1: 'async_trait, 'life2: 'async_trait, Self: 'async_trait,

Called after a tool completes successfully.

fn on_tool_error<'life0, 'life1, 'life2, 'async_trait>( &'life0 self, _call: &'life1 FunctionCall, _err: &'life2 ToolError, ) -> Pin<Box<dyn Future<Output = Result<(), AgentError>> + Send + 'async_trait>>

where 'life0: 'async_trait, 'life1: 'async_trait, 'life2: 'async_trait, Self: 'async_trait,

Called when a tool execution fails.

fn on_event<'life0, 'life1, 'async_trait>( &'life0 self, _event: &'life1 AgentEvent, ) -> Pin<Box<dyn Future<Output = Result<(), AgentError>> + Send + 'async_trait>>

where 'life0: 'async_trait, 'life1: 'async_trait, Self: 'async_trait,

Called when an agent event is emitted.

fn on_error<'life0, 'life1, 'async_trait>( &'life0 self, _err: &'life1 AgentError, ) -> Pin<Box<dyn Future<Output = Result<(), AgentError>> + Send + 'async_trait>>

where 'life0: 'async_trait, 'life1: 'async_trait, Self: 'async_trait,

Called when an agent error occurs.

fn transform_request<'life0, 'life1, 'async_trait>( &'life0 self, _request: &'life1 mut LlmRequest, ) -> Pin<Box<dyn Future<Output = Result<(), AgentError>> + Send + 'async_trait>>

where 'life0: 'async_trait, 'life1: 'async_trait, Self: 'async_trait,

Called with the fully-built request before it is sent to the model, allowing in-place mutation (e.g. trimming or rewriting conversation history). Runs ahead of Middleware::before_model. This mirrors the mutable before_model_callback request hook in the ADK Python SDK.

fn timeout(&self) -> Option<Duration>

Maximum wall-clock duration this middleware imposes on the agent run. None (the default) means no limit. The agent enforces the tightest timeout across its middleware chain by bounding the whole run.