Struct UserSimulatorEvaluator 

pub struct UserSimulatorEvaluator { /* private fields */ }

Evaluates the fidelity of a user simulator in multi-turn conversations.

Assesses whether simulated user messages are:

• Realistic and coherent
• Following the assigned persona/scenario
• Providing adequate coverage of the test scenario
• Properly using the stop signal when the conversation should end

Implementations

impl UserSimulatorEvaluator

pub fn new() -> Self

Create a new user simulator evaluator.

pub fn with_stop_signal(self, signal: impl Into<String>) -> Self

Set the stop signal that the simulator uses to end conversations.

pub fn with_judge_model(self, model: impl Into<String>) -> Self

Set an override judge model name.

pub fn with_llm(self, llm: Arc<dyn BaseLlm>) -> Self

Provide an LLM instance for performing evaluations.

Trait Implementations

impl Default for UserSimulatorEvaluator

fn default() -> Self

Returns the “default value” for a type.

impl Evaluator for UserSimulatorEvaluator

fn evaluate<'life0, 'life1, 'life2, 'async_trait>( &'life0 self, actual: &'life1 [Invocation], _expected: Option<&'life2 [Invocation]>, ) -> Pin<Box<dyn Future<Output = Result<EvalResult, EvalError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait, 'life2: 'async_trait,

Evaluate agent invocations.