pub struct TapedLlm<L = Arc<dyn BaseLlm>> { /* private fields */ }Expand description
A BaseLlm that records its calls to a Tape or answers them from
one. See the module docs.
Implementations§
Trait Implementations§
Source§impl<L> BaseLlm for TapedLlm<L>where
L: BaseLlm,
impl<L> BaseLlm for TapedLlm<L>where
L: BaseLlm,
Source§fn capabilities(&self) -> ModelCapabilities
fn capabilities(&self) -> ModelCapabilities
What this model supports — the ADK model-capability-declaration
pattern: the model states its capabilities instead of every caller
re-inferring them from the id string. The default derives a
conservative estimate from
model_id; back-ends
with authoritative knowledge should override.Source§fn generate<'life0, 'async_trait>(
&'life0 self,
request: LlmRequest,
) -> Pin<Box<dyn Future<Output = Result<LlmResponse, LlmError>> + Send + 'async_trait>>where
'life0: 'async_trait,
TapedLlm<L>: 'async_trait,
fn generate<'life0, 'async_trait>(
&'life0 self,
request: LlmRequest,
) -> Pin<Box<dyn Future<Output = Result<LlmResponse, LlmError>> + Send + 'async_trait>>where
'life0: 'async_trait,
TapedLlm<L>: 'async_trait,
Generate content from the LLM.
Source§fn warm_up<'life0, 'async_trait>(
&'life0 self,
) -> Pin<Box<dyn Future<Output = Result<(), LlmError>> + Send + 'async_trait>>where
'life0: 'async_trait,
TapedLlm<L>: 'async_trait,
fn warm_up<'life0, 'async_trait>(
&'life0 self,
) -> Pin<Box<dyn Future<Output = Result<(), LlmError>> + Send + 'async_trait>>where
'life0: 'async_trait,
TapedLlm<L>: 'async_trait,
Pre-warm the HTTP connection pool to avoid cold-start latency. Read more
Source§fn generate_stream<'life0, 'async_trait>(
&'life0 self,
request: LlmRequest,
) -> Pin<Box<dyn Future<Output = Result<Pin<Box<dyn Stream<Item = Result<LlmResponse, LlmError>> + Send>>, LlmError>> + Send + 'async_trait>>where
'life0: 'async_trait,
Self: 'async_trait,
fn generate_stream<'life0, 'async_trait>(
&'life0 self,
request: LlmRequest,
) -> Pin<Box<dyn Future<Output = Result<Pin<Box<dyn Stream<Item = Result<LlmResponse, LlmError>> + Send>>, LlmError>> + Send + 'async_trait>>where
'life0: 'async_trait,
Self: 'async_trait,
Generate content as a stream of chunks, in the order the model
produced them.
LlmResponse::append folds them into the whole reply;
usage, when a chunk carries it, is cumulative. Read moreAuto Trait Implementations§
impl<L> Freeze for TapedLlm<L>where
L: Freeze,
impl<L = Arc<dyn BaseLlm>> !RefUnwindSafe for TapedLlm<L>
impl<L> Send for TapedLlm<L>where
L: Send,
impl<L> Sync for TapedLlm<L>where
L: Sync,
impl<L> Unpin for TapedLlm<L>where
L: Unpin,
impl<L = Arc<dyn BaseLlm>> !UnwindSafe for TapedLlm<L>
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more
§impl<S> FromSample<S> for S
impl<S> FromSample<S> for S
fn from_sample_(s: S) -> S
§impl<T> FutureExt for T
impl<T> FutureExt for T
§fn with_context(self, otel_cx: Context) -> WithContext<Self>
fn with_context(self, otel_cx: Context) -> WithContext<Self>
§fn with_current_context(self) -> WithContext<Self>
fn with_current_context(self) -> WithContext<Self>
§impl<T> Instrument for T
impl<T> Instrument for T
§fn instrument(self, span: Span) -> Instrumented<Self>
fn instrument(self, span: Span) -> Instrumented<Self>
§fn in_current_span(self) -> Instrumented<Self>
fn in_current_span(self) -> Instrumented<Self>
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self>
fn into_either(self, into_left: bool) -> Either<Self, Self>
Converts
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
Converts
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read more§impl<T> IntoRequest<T> for T
impl<T> IntoRequest<T> for T
§fn into_request(self) -> Request<T>
fn into_request(self) -> Request<T>
Wrap the input message
T in a tonic::Request§impl<L> LayerExt<L> for L
impl<L> LayerExt<L> for L
§fn named_layer<S>(&self, service: S) -> Layered<<L as Layer<S>>::Service, S>where
L: Layer<S>,
fn named_layer<S>(&self, service: S) -> Layered<<L as Layer<S>>::Service, S>where
L: Layer<S>,
Applies the layer to a service and wraps it in [
Layered].