estimate_tokens

Function estimate_tokens 

Source
pub fn estimate_tokens(text: &str) -> usize
Expand description

Approximate the token cost of a string.

Deliberately an over-estimate: the budget exists to protect latency and context, and being slightly conservative is cheaper than blowing the cap.