gemini_adk_fluent_rs/voice/dsp/aec.rs
1//! Acoustic echo cancellation: subtract the bot's own voice from the mic
2//! before anything else touches it.
3//!
4//! # Why this stage exists
5//!
6//! On an open-speaker device (phone speakerphone, conference room, a laptop
7//! with no headset) the bot's own synthesized voice leaves the speaker and
8//! re-enters the microphone a few milliseconds to a few hundred milliseconds
9//! later, attenuated and reshaped by the room. Under client-interruption
10//! authority (the model treats any mic energy as "the user is talking, stop")
11//! that echo makes the bot interrupt itself mid-sentence. The fix is not a
12//! bigger VAD threshold — it is removing the echo from the signal the VAD
13//! (and the ASR, and the model) ever sees.
14//!
15//! The far-end reference — the audio actually handed to the speaker — is
16//! available in this SDK on the playback path. [`Aec::new`] returns an
17//! [`AecFarEnd`] handle; feed it the same PCM the speaker plays
18//! ([`AecFarEnd::push_pcm16`] / [`AecFarEnd::push_f32`]) and this stage
19//! predicts and subtracts the echo before it reaches the rest of the mic
20//! chain. **This stage must run before the denoiser** — RNNoise (and any
21//! other nonlinear enhancer) rewrites the spectrum in ways that break the
22//! linear room-response model the adaptive filter is trying to learn; feed
23//! it a denoised signal and it converges on garbage, if it converges at all.
24//!
25//! # Algorithm: partitioned-block frequency-domain NLMS (overlap-save)
26//!
27//! This is a linear echo canceller — it models the echo path as an FIR
28//! filter of `tail_ms` and adapts that filter with normalized LMS, done in
29//! the frequency domain per 10 ms block for efficiency (one FFT pair
30//! per block handles a filter tail hundreds of taps long). It does **not**
31//! handle nonlinear echo (cheap-speaker clipping/distortion) — that needs a
32//! nonlinear residual-echo suppressor layered after this stage, out of scope
33//! here.
34//!
35//! - Block size `B` = 10 ms of samples (160 at 16 kHz). FFT size
36//! `N = 2B` (overlap-save discipline: a linear `B`-tap convolution result
37//! is only valid in the *last* `B` samples of a `2B`-point circular
38//! convolution, so every inverse transform here keeps only that half and
39//! discards the first `B` as circular-wraparound garbage).
40//! - The echo path is split into `P = ceil(tail_ms / 10ms)` partitions of
41//! `B` taps each, one weight vector `W_p` (complex, `B+1` one-sided bins)
42//! per partition. Only **one** forward FFT of the far-end is computed per
43//! block; the `P` per-partition spectra are a rolling history of that same
44//! transform (a ring buffer), not `P` separate transforms — this is the
45//! entire point of "partitioned" convolution.
46//! - Echo estimate: `Y = Σ_p W_p ⊙ X_p`; the time-domain estimate is the last
47//! `B` samples of `IFFT(Y)`, scaled by `1/N` (`realfft`/`rustfft` transforms
48//! are unnormalized in both directions, so a forward+inverse round trip
49//! scales amplitude by `N` unless corrected — see the source for where
50//! that correction lands).
51//! - NLMS update in the frequency domain: `W_p[k] += μ · conj(X_p[k]) ·
52//! E[k] / (Px[k] + ε)`, where `E = FFT([zeros(B), e])` (error zero-padded
53//! at the *front* — the adjoint of "keep only the last `B` samples" used
54//! to form the estimate) and `Px[k]` is an EWMA (0.9 retained / 0.1 new,
55//! the same convention [`DspChain`](super::DspChain) uses for its meters)
56//! of `Σ_p |X_p[k]|²`.
57//! - Gradient constraint, applied every block: IFFT each just-updated `W_p`
58//! back to time domain, zero the *last* `B` samples (an unconstrained
59//! frequency-domain update can grow acausal/wraparound content that a
60//! real `B`-tap filter can't have), FFT back. At this block size the cost
61//! (`2P` extra transforms/block) is cheap enough to just always pay it.
62//!
63//! # Double-talk protection
64//!
65//! Adapting while the near-end user is also speaking teaches the filter to
66//! partially cancel the user, which is exactly backwards. Gating is cheap
67//! and Geigel-style: adaptation only runs when the far-end block power
68//! exceeds a floor (there is nothing to learn from if the bot is silent)
69//! *and* the mic peak for the block does not exceed `0.9 ×` the largest
70//! far-end peak seen in the last `P` blocks — a mic level comparable to (or
71//! louder than) the recent far-end implies near-end speech is present, since
72//! real echo return loss attenuates. A trip freezes adaptation (not echo
73//! cancellation — the existing filter keeps subtracting its prediction) for
74//! a ~30-block (~300 ms) hangover so a single loud consonant doesn't
75//! cause the filter to start re-adapting mid-sentence.
76//!
77//! # Bulk delay
78//!
79//! `delay_ms` (default 40) compensates the latency between "audio handed to
80//! [`AecFarEnd`]" and "that audio arrives at the mic via air/room path" —
81//! mostly playback buffering, not room propagation. It is implemented as a
82//! FIFO the far-end reference passes through before it ever reaches the
83//! filter. **`tail_ms` must cover whatever misalignment remains** after this
84//! coarse compensation (clock drift, a `delay_ms` that's an estimate rather
85//! than measured, etc.) — the adaptive filter can only pull in echo that
86//! falls inside its `P`-partition window relative to the delayed reference;
87//! echo arriving *earlier* than the (delayed) reference cannot be modeled by
88//! a causal filter at all, and echo arriving later than `tail_ms` past it
89//! is simply not learned.
90//!
91//! # Known failure modes (stated, not hidden)
92//!
93//! - **Far-end underrun**: if [`AecFarEnd`] hasn't been fed enough audio to
94//! fill the delay line for a given mic block, the missing far-end samples
95//! are treated as silence. No echo is predicted for that block — the raw
96//! (uncancelled) mic audio passes through for whatever portion is missing.
97//! - **Startup**: for the first `delay_ms` worth of audio the delay line is
98//! still draining its zero-fill primer, so there is nothing to cancel yet
99//! even if far-end audio is already flowing.
100//! - **This stage always adds exactly one block (`B` samples) of latency**,
101//! declared honestly via [`DspStage::latency_samples`] — mic audio is
102//! buffered internally until a full block is available, processed, and
103//! the *previous* block's result is what comes out, so arbitrary input
104//! chunk sizes are supported (a stage caller need not chunk to `B` itself)
105//! at the cost of that fixed one-block delay.
106//! - **Mono, single far-end source only.** Stereo far-end / multiple
107//! simultaneous playback streams are out of scope.
108
109use std::collections::VecDeque;
110use std::sync::Arc;
111
112use parking_lot::Mutex;
113use realfft::num_complex::Complex32;
114use realfft::{ComplexToReal, RealFftPlanner, RealToComplex};
115
116use super::{AudioBus, DspStage};
117
118/// Far-end block power below this is "the bot isn't talking" — don't adapt,
119/// there is nothing to learn an echo path from.
120const FAR_ACTIVE_THRESHOLD: f32 = 1e-6;
121/// Geigel double-talk ratio: mic peak vs. the recent far-end peak.
122const GEIGEL_RATIO: f32 = 0.9;
123/// Blocks to keep adaptation frozen after a double-talk trip (~300 ms at
124/// the 10 ms block size).
125const HANGOVER_BLOCKS: u32 = 30;
126/// NLMS regularization floor — prevents division blow-up when the far-end
127/// spectrum is near zero. Negligible next to any real signal's power.
128const EPS: f32 = 1e-6;
129/// EWMA retain weight for the `Px` power estimate and the ERLE meters —
130/// matches the smoothing convention already used by
131/// [`DspChain`](super::DspChain)'s stage meters.
132const EWMA_RETAIN: f32 = 0.9;
133/// How much far-end audio [`AecFarEnd`] buffers before dropping the oldest
134/// samples — bounds memory if the far-end producer runs ahead of the mic
135/// consumer indefinitely.
136const FAR_QUEUE_SECONDS: usize = 2;
137
138/// Tunables for [`Aec::new`]. All three fields have the module's tested
139/// defaults via [`Default`].
140pub struct AecConfig {
141 /// Length of the modeled echo tail, in milliseconds. Rounded up to a
142 /// whole number of 10 ms partitions. Must cover the room's actual
143 /// reverberant echo plus any residual misalignment left after
144 /// `delay_ms` — see the module docs on bulk delay.
145 pub tail_ms: u32,
146 /// Bulk delay compensation applied to the far-end reference before it
147 /// enters the filter, in milliseconds — models playback pipeline
148 /// latency between "handed to the speaker" and "captured by the mic".
149 pub delay_ms: u32,
150 /// NLMS step size. The theoretical bound for a power-normalized
151 /// update is `0 < mu < 2`; the practical bound on narrowband bursty
152 /// far ends (speech through a speaker) measured far below it:
153 /// 0.25+ diverges within seconds, 0.1 reaches a bad marginal
154 /// equilibrium (gradient-noise misadjustment above the echo level),
155 /// 0.05 holds through every measured stress with ERLE +8..11 dB on
156 /// the worst-case harmonic proxy and 12+ dB on broadband far ends.
157 /// Default 0.05 — raise it only with `evals/dspbench` watching.
158 pub mu: f32,
159}
160
161impl Default for AecConfig {
162 fn default() -> Self {
163 Self {
164 tail_ms: 128,
165 delay_ms: 40,
166 mu: 0.05,
167 }
168 }
169}
170
171/// Handle the playback side feeds with the same audio being sent to the
172/// speaker. Cheap to clone (shares one queue via `Arc`); safe to call from
173/// a different task/thread than the one driving [`Aec::process`].
174#[derive(Clone)]
175pub struct AecFarEnd {
176 queue: Arc<Mutex<VecDeque<f32>>>,
177 max_len: usize,
178}
179
180impl AecFarEnd {
181 /// Push PCM16 samples (converted to `f32` by `/32768`), oldest dropped
182 /// first if the internal buffer is over its bound.
183 pub fn push_pcm16(&self, samples: &[i16]) {
184 let mut q = self.queue.lock();
185 for &s in samples {
186 push_bounded(&mut q, self.max_len, f32::from(s) / 32768.0);
187 }
188 }
189
190 /// Push `f32` samples directly (nominal `[-1.0, 1.0]`), oldest dropped
191 /// first if the internal buffer is over its bound.
192 pub fn push_f32(&self, samples: &[f32]) {
193 let mut q = self.queue.lock();
194 for &s in samples {
195 push_bounded(&mut q, self.max_len, s);
196 }
197 }
198}
199
200fn push_bounded(q: &mut VecDeque<f32>, max_len: usize, sample: f32) {
201 if q.len() >= max_len {
202 q.pop_front();
203 }
204 q.push_back(sample);
205}
206
207/// Partitioned-block frequency-domain NLMS acoustic echo canceller. See the
208/// module docs for the algorithm and its stated limits.
209pub struct Aec {
210 sample_rate: u32,
211 /// Block size `B`, in samples (10 ms).
212 block: usize,
213 /// FFT size `N = 2B`.
214 fft_len: usize,
215 /// Number of `B`-tap partitions covering `tail_ms`.
216 partitions: usize,
217 mu: f32,
218
219 r2c: Arc<dyn RealToComplex<f32>>,
220 c2r: Arc<dyn ComplexToReal<f32>>,
221 r2c_scratch: Vec<Complex32>,
222 c2r_scratch: Vec<Complex32>,
223 /// Reusable time-domain scratch, length `fft_len`. Contents are garbage
224 /// between steps within a block — never read without first being
225 /// written in the same block.
226 time_scratch: Vec<f32>,
227
228 /// Ring buffer of the far-end spectrum, one slot per partition; `head`
229 /// names the slot holding the newest (`X_0`) transform.
230 x_hist: Vec<Vec<Complex32>>,
231 head: usize,
232 /// Adaptive filter weights, one vector per logical partition (fixed
233 /// indexing — `w[p]` always means "coefficients for lag `p` blocks",
234 /// unlike `x_hist` which rotates).
235 w: Vec<Vec<Complex32>>,
236 /// EWMA of the far-end power spectrum, `Σ_p |X_p[k]|²`.
237 px: Vec<f32>,
238 /// Previous far-end block, for building the overlap-save `[prev, cur]`
239 /// window.
240 far_hist_prev: Vec<f32>,
241 y_freq: Vec<Complex32>,
242 e_freq: Vec<Complex32>,
243
244 mic_block: Vec<f32>,
245 far_block: Vec<f32>,
246 error: Vec<f32>,
247
248 in_fifo: VecDeque<f32>,
249 out_fifo: VecDeque<f32>,
250 far_queue: Arc<Mutex<VecDeque<f32>>>,
251 far_delay_buf: VecDeque<f32>,
252
253 far_peak_hist: VecDeque<f32>,
254 hangover: u32,
255
256 erle_ever_active: bool,
257 erle_mic_pow: f32,
258 erle_err_pow: f32,
259}
260
261impl Aec {
262 /// Build the canceller and its paired far-end feed handle. `sample_rate`
263 /// must match the [`AudioBus`] this stage will be run on.
264 pub fn new(config: AecConfig, sample_rate: u32) -> (Self, AecFarEnd) {
265 let block = ((u64::from(sample_rate) * 10 / 1000) as usize).max(1);
266 let fft_len = block * 2;
267 let bins = block + 1;
268
269 let tail_samples = u64::from(config.tail_ms) * u64::from(sample_rate) / 1000;
270 let partitions = ((tail_samples as f64 / block as f64).ceil() as usize).max(1);
271 let delay_samples = (u64::from(config.delay_ms) * u64::from(sample_rate) / 1000) as usize;
272
273 let mut planner = RealFftPlanner::<f32>::new();
274 let r2c = planner.plan_fft_forward(fft_len);
275 let c2r = planner.plan_fft_inverse(fft_len);
276 let r2c_scratch = r2c.make_scratch_vec();
277 let c2r_scratch = c2r.make_scratch_vec();
278
279 let zero_bins = || vec![Complex32::new(0.0, 0.0); bins];
280 let x_hist = (0..partitions).map(|_| zero_bins()).collect();
281 let w = (0..partitions).map(|_| zero_bins()).collect();
282
283 let mut out_fifo = VecDeque::with_capacity(block * 2);
284 out_fifo.extend(std::iter::repeat_n(0.0f32, block));
285
286 let mut far_delay_buf = VecDeque::with_capacity(delay_samples + block);
287 far_delay_buf.extend(std::iter::repeat_n(0.0f32, delay_samples));
288
289 let far_queue = Arc::new(Mutex::new(VecDeque::new()));
290 let max_len = sample_rate as usize * FAR_QUEUE_SECONDS;
291
292 let aec = Self {
293 sample_rate,
294 block,
295 fft_len,
296 partitions,
297 mu: config.mu,
298 r2c,
299 c2r,
300 r2c_scratch,
301 c2r_scratch,
302 time_scratch: vec![0.0; fft_len],
303 x_hist,
304 head: 0,
305 w,
306 px: vec![0.0; bins],
307 far_hist_prev: vec![0.0; block],
308 y_freq: zero_bins(),
309 e_freq: zero_bins(),
310 mic_block: Vec::with_capacity(block),
311 far_block: Vec::with_capacity(block),
312 error: Vec::with_capacity(block),
313 in_fifo: VecDeque::new(),
314 out_fifo,
315 far_queue: Arc::clone(&far_queue),
316 far_delay_buf,
317 far_peak_hist: VecDeque::with_capacity(partitions),
318 hangover: 0,
319 erle_ever_active: false,
320 erle_mic_pow: 0.0,
321 erle_err_pow: 0.0,
322 };
323 (
324 aec,
325 AecFarEnd {
326 queue: far_queue,
327 max_len,
328 },
329 )
330 }
331
332 /// Echo Return Loss Enhancement, in dB: `10·log10(EWMA(mic power) /
333 /// EWMA(error power))`, measured only over blocks where the far-end was
334 /// active. `0.0` before any far-end activity has ever been observed —
335 /// there is nothing yet to report a ratio over.
336 pub fn erle_db(&self) -> f32 {
337 if !self.erle_ever_active {
338 return 0.0;
339 }
340 10.0 * (self.erle_mic_pow.max(1e-20) / self.erle_err_pow.max(1e-20)).log10()
341 }
342
343 fn process_block(&mut self) {
344 let block = self.block;
345 let n = self.fft_len;
346 let r2c = Arc::clone(&self.r2c);
347 let c2r = Arc::clone(&self.c2r);
348
349 // 1. Pull one block of mic samples and the delayed far-end reference.
350 self.mic_block.clear();
351 self.mic_block.extend(self.in_fifo.drain(0..block));
352
353 {
354 let mut q = self.far_queue.lock();
355 self.far_delay_buf.extend(q.drain(..));
356 }
357 let avail = self.far_delay_buf.len().min(block);
358 self.far_block.clear();
359 self.far_block.extend(self.far_delay_buf.drain(0..avail));
360 // Underrun: missing far-end samples are silence (no echo predicted
361 // for them) — see the module docs' failure-modes section.
362 self.far_block.resize(block, 0.0);
363
364 // 2. Overlap-save window [prev B, cur B] -> one forward FFT, stored
365 // as the newest ring slot (X_0).
366 self.time_scratch[..block].copy_from_slice(&self.far_hist_prev);
367 self.time_scratch[block..].copy_from_slice(&self.far_block);
368 self.head = (self.head + 1) % self.partitions;
369 r2c.process_with_scratch(
370 &mut self.time_scratch,
371 &mut self.x_hist[self.head],
372 &mut self.r2c_scratch,
373 )
374 .expect("aec: far-end forward fft (fixed sizes)");
375 self.far_hist_prev.copy_from_slice(&self.far_block);
376
377 // 3. Echo estimate Y = sum_p W_p * X_p; keep the last B samples of
378 // IFFT(Y)/N (overlap-save: the first B are circular garbage).
379 for c in &mut self.y_freq {
380 *c = Complex32::new(0.0, 0.0);
381 }
382 for p in 0..self.partitions {
383 let idx = (self.head + self.partitions - p) % self.partitions;
384 let xp = &self.x_hist[idx];
385 let wp = &self.w[p];
386 for ((y, w), x) in self.y_freq.iter_mut().zip(wp.iter()).zip(xp.iter()) {
387 *y += *w * *x;
388 }
389 }
390 c2r.process_with_scratch(
391 &mut self.y_freq,
392 &mut self.time_scratch,
393 &mut self.c2r_scratch,
394 )
395 .expect("aec: echo inverse fft (fixed sizes)");
396 let norm = 1.0 / n as f32;
397
398 // 4. Error = mic - echo estimate. This is the stage's output.
399 let echo_tail = &self.time_scratch[block..];
400 self.error.clear();
401 self.error.extend(
402 self.mic_block
403 .iter()
404 .zip(echo_tail.iter())
405 .map(|(&mic, &y)| mic - y * norm),
406 );
407 let mic_pow = mean_power(&self.mic_block);
408 let err_pow = mean_power(&self.error);
409
410 // 5. Error spectrum for the gradient: E = FFT([zeros(B), e]) — the
411 // zero-padding-at-the-front is the adjoint of step 3's "keep only
412 // the last B samples".
413 self.time_scratch[..block].fill(0.0);
414 self.time_scratch[block..].copy_from_slice(&self.error);
415 r2c.process_with_scratch(
416 &mut self.time_scratch,
417 &mut self.e_freq,
418 &mut self.r2c_scratch,
419 )
420 .expect("aec: error forward fft (fixed sizes)");
421
422 // 6. Px EWMA of the far-end power spectrum (NLMS normalizer).
423 // Held frozen while the far end is silent: decaying it through
424 // inter-burst gaps would make the next burst onset see a fraction
425 // of the true power and overshoot the NLMS stability bound —
426 // measured as divergence on speech-like (bursty) far ends.
427 let far_active = mean_power(&self.far_block) > FAR_ACTIVE_THRESHOLD;
428 if far_active {
429 for px in &mut self.px {
430 *px *= EWMA_RETAIN;
431 }
432 for p in 0..self.partitions {
433 let idx = (self.head + self.partitions - p) % self.partitions;
434 let xp = &self.x_hist[idx];
435 for (px, x) in self.px.iter_mut().zip(xp.iter()) {
436 *px += (1.0 - EWMA_RETAIN) * x.norm_sqr();
437 }
438 }
439 }
440
441 // 7. Far-end-activity + Geigel double-talk gating.
442 let far_peak = self.far_block.iter().fold(0.0f32, |m, &s| m.max(s.abs()));
443 let mic_peak = self.mic_block.iter().fold(0.0f32, |m, &s| m.max(s.abs()));
444
445 if self.far_peak_hist.len() == self.partitions {
446 self.far_peak_hist.pop_front();
447 }
448 self.far_peak_hist.push_back(far_peak);
449 let max_far_peak = self.far_peak_hist.iter().fold(0.0f32, |m, &s| m.max(s));
450
451 let double_talk = far_active && mic_peak > GEIGEL_RATIO * max_far_peak;
452 if double_talk {
453 self.hangover = HANGOVER_BLOCKS;
454 }
455 let adapting = far_active && self.hangover == 0;
456 if self.hangover > 0 {
457 self.hangover -= 1;
458 }
459
460 // 8. NLMS update, gated by double-talk/far-end-activity.
461 if adapting {
462 let mu = self.mu;
463 // Leaky NLMS: damp weight components the far end never
464 // excites. Narrowband input leaves most bins unexcited; energy
465 // accumulating there from error noise is what turned marginal
466 // stability into a bad equilibrium (output above mic) on
467 // sustained harmonic far ends. ~0.1% decay per 10 ms block is
468 // invisible to converged echo paths (they are re-excited every
469 // block) and, together with the unexcited-bin gate below, fatal
470 // to the parasitic modes. (0.1%/block measured too strong: it
471 // capped converged broadband ERLE at ~9 dB.)
472 const LEAK: f32 = 0.9999;
473 for wp in &mut self.w {
474 for w in wp.iter_mut() {
475 *w *= LEAK;
476 }
477 }
478 // Spectral regularization: floor every bin's normalizer at 1%
479 // of the mean bin power. In bins where a narrowband far end
480 // has no energy, Px is microscopic and the update would divide
481 // by EPS alone — the weights there random-walk on error noise
482 // until feedback diverges (measured on harmonic far ends).
483 let px_mean = self.px.iter().sum::<f32>() / self.px.len().max(1) as f32;
484 let px_floor = px_mean * 0.01;
485 for p in 0..self.partitions {
486 let idx = (self.head + self.partitions - p) % self.partitions;
487 let xp = &self.x_hist[idx];
488 let wp = &mut self.w[p];
489 let updates = wp
490 .iter_mut()
491 .zip(xp.iter())
492 .zip(self.px.iter())
493 .zip(self.e_freq.iter());
494 for (((w, x), px), e) in updates {
495 // Bins the far end doesn't excite get NO update at all
496 // (their weights only leak toward zero): with narrowband
497 // input, updating unexcited bins lets error noise
498 // accumulate parasitic weight energy that ends up
499 // louder than the echo it was meant to cancel.
500 if *px < px_floor {
501 continue;
502 }
503 // Floor the normalizer with this bin's instantaneous
504 // power: whatever the EWMA lags, the effective step
505 // stays <= mu, inside the NLMS stability bound.
506 let denom = px.max(x.norm_sqr()) + EPS;
507 let grad = x.conj() * *e;
508 let mut updated = *w + grad * (mu / denom);
509 if !updated.re.is_finite() || !updated.im.is_finite() {
510 updated = Complex32::new(0.0, 0.0);
511 }
512 *w = updated;
513 }
514 }
515 }
516
517 // 9. Gradient constraint (every block): project each partition back
518 // onto a causal B-tap filter so an unconstrained frequency-domain
519 // step can't grow acausal/wraparound content.
520 for p in 0..self.partitions {
521 c2r.process_with_scratch(
522 &mut self.w[p],
523 &mut self.time_scratch,
524 &mut self.c2r_scratch,
525 )
526 .expect("aec: constraint inverse fft (fixed sizes)");
527 for s in &mut self.time_scratch {
528 *s *= norm;
529 }
530 for s in &mut self.time_scratch[block..] {
531 *s = 0.0;
532 }
533 r2c.process_with_scratch(
534 &mut self.time_scratch,
535 &mut self.w[p],
536 &mut self.r2c_scratch,
537 )
538 .expect("aec: constraint forward fft (fixed sizes)");
539 }
540
541 // 10. ERLE bookkeeping — only on blocks where cancellation is
542 // measurable: far end active AND no double-talk (during double
543 // talk the error carries near-end speech by design; feeding those
544 // blocks poisons the meter for many seconds afterwards).
545 if adapting {
546 // Slow meter (~2 s memory): the fast Px constant would let
547 // burst onsets and AM valleys drag the reading far below the
548 // converged cancellation the waveforms show.
549 const ERLE_RETAIN: f32 = 0.995;
550 if self.erle_ever_active {
551 self.erle_mic_pow = self.erle_mic_pow * ERLE_RETAIN + mic_pow * (1.0 - ERLE_RETAIN);
552 self.erle_err_pow = self.erle_err_pow * ERLE_RETAIN + err_pow * (1.0 - ERLE_RETAIN);
553 } else {
554 self.erle_mic_pow = mic_pow;
555 self.erle_err_pow = err_pow;
556 self.erle_ever_active = true;
557 }
558 }
559 self.out_fifo.extend(self.error.iter().copied());
560 }
561}
562
563fn mean_power(samples: &[f32]) -> f32 {
564 if samples.is_empty() {
565 return 0.0;
566 }
567 let energy: f64 = samples.iter().map(|&s| f64::from(s) * f64::from(s)).sum();
568 (energy / samples.len() as f64) as f32
569}
570
571impl DspStage for Aec {
572 fn name(&self) -> &'static str {
573 "aec"
574 }
575
576 fn process(&mut self, bus: &mut AudioBus) {
577 debug_assert_eq!(
578 bus.sample_rate, self.sample_rate,
579 "Aec was built for a different sample rate than the bus it's running on"
580 );
581 let want = bus.samples.len();
582 self.in_fifo.extend(bus.samples.iter().copied());
583 while self.in_fifo.len() >= self.block {
584 self.process_block();
585 }
586 debug_assert!(self.out_fifo.len() >= want, "aec output FIFO underrun");
587 bus.samples.clear();
588 bus.samples.extend(self.out_fifo.drain(0..want));
589 }
590
591 fn latency_samples(&self) -> usize {
592 self.block
593 }
594}
595
596#[cfg(test)]
597mod tests {
598 #[test]
599 fn stays_stable_on_narrowband_bursty_far_end() {
600 // The discriminating input the white-noise tests miss: a harmonic,
601 // bursty far end (speech through a speaker). Three separate
602 // instabilities were measured here before their fixes: burst-onset
603 // step overshoot (decaying Px), empty-bin amplification (Px ~ 0 in
604 // bins the tone never touches), and a step size above the
605 // practical narrowband bound.
606 let sr = 16_000u32;
607 let n = 20 * sr as usize;
608 let mut far = vec![0.0f32; n];
609 let (on, off) = (sr as usize * 16 / 10, sr as usize * 6 / 10);
610 let mut i = 0usize;
611 let mut f0 = 120.0f32;
612 while i < n {
613 let end = (i + on).min(n);
614 for (k, slot) in far[i..end].iter_mut().enumerate() {
615 let t = k as f32 / sr as f32;
616 let am = 0.6 + 0.4 * (2.0 * std::f32::consts::PI * 4.0 * t).sin();
617 let mut v = 0.0;
618 for h in 1..=4 {
619 v += (2.0 * std::f32::consts::PI * f0 * h as f32 * t).sin() / h as f32;
620 }
621 *slot = 0.12 * am * v;
622 }
623 f0 = if f0 > 190.0 { 120.0 } else { f0 + 23.0 };
624 i = end + off;
625 }
626 // Echo path: sparse decaying taps, 40 ms bulk delay.
627 let taps: [f32; 6] = [0.12, -0.05, 0.03, -0.015, 0.008, -0.004];
628 let delay = 40 * sr as usize / 1000;
629 let mut mic = vec![0.0f32; n];
630 for (j, m) in mic.iter_mut().enumerate() {
631 for (ti, &tap) in taps.iter().enumerate() {
632 let src = j as isize - delay as isize - (ti as isize * 37);
633 if src >= 0 {
634 *m += tap * far[src as usize];
635 }
636 }
637 }
638
639 let (mut aec, far_end) = Aec::new(
640 AecConfig {
641 delay_ms: 40,
642 ..AecConfig::default()
643 },
644 sr,
645 );
646 let mut out = Vec::with_capacity(n);
647 for (idx, block) in mic.chunks(320).enumerate() {
648 let a = idx * 320;
649 far_end.push_f32(&far[a..(a + block.len()).min(n)]);
650 let mut buf = block.to_vec();
651 let mut bus = AudioBus {
652 samples: &mut buf,
653 sample_rate: sr,
654 };
655 aec.process(&mut bus);
656 out.extend_from_slice(&buf);
657 }
658
659 assert!(out.iter().all(|s| s.is_finite()), "output diverged");
660 let tail = &out[n - 2 * sr as usize..];
661 let mic_tail = &mic[n - 2 * sr as usize..];
662 let rms = |x: &[f32]| (x.iter().map(|s| s * s).sum::<f32>() / x.len() as f32).sqrt();
663 assert!(
664 rms(tail) < 0.5 * rms(mic_tail),
665 "no cancellation: out {} vs mic {}",
666 rms(tail),
667 rms(mic_tail)
668 );
669 assert!(aec.erle_db() > 6.0, "erle {} dB", aec.erle_db());
670 }
671
672 use super::*;
673
674 /// Fixed-seed xorshift32 — deterministic, no `rand` dependency.
675 struct Xorshift32(u32);
676
677 impl Xorshift32 {
678 fn new(seed: u32) -> Self {
679 Self(seed.max(1))
680 }
681
682 fn next_u32(&mut self) -> u32 {
683 let mut x = self.0;
684 x ^= x << 13;
685 x ^= x >> 17;
686 x ^= x << 5;
687 self.0 = x;
688 x
689 }
690
691 /// Uniform in `[-1.0, 1.0)`.
692 fn next_signed(&mut self) -> f32 {
693 (self.next_u32() as f32 / u32::MAX as f32).mul_add(2.0, -1.0)
694 }
695 }
696
697 const SR: u32 = 16_000;
698 const BLOCK: usize = 160;
699
700 /// 48-tap synthetic room response: decaying, sign-alternating every 4
701 /// taps (`0.05 * (-0.7)^(i/4)`). Scaled to `0.05` rather than the naive
702 /// `0.5` first-tap gain: repeating each decay step over 4 taps before
703 /// attenuating means the *sum* of the naive-amplitude taps exceeds unity
704 /// gain (a "louder than the far end" echo), which is not physically
705 /// realistic (echo return loss attenuates) and — more importantly for
706 /// the test — would spuriously trip the Geigel double-talk detector on
707 /// pure echo with no near-end speech at all.
708 fn rir_taps() -> Vec<f32> {
709 (0..48).map(|i: i32| 0.05 * (-0.7f32).powi(i / 4)).collect()
710 }
711
712 fn convolve_causal(far: &[f32], h: &[f32]) -> Vec<f32> {
713 let mut out = vec![0.0f32; far.len()];
714 for (n, out_n) in out.iter_mut().enumerate() {
715 let mut acc = 0.0f32;
716 for (i, &hi) in h.iter().enumerate() {
717 if i > n {
718 break;
719 }
720 acc += hi * far[n - i];
721 }
722 *out_n = acc;
723 }
724 out
725 }
726
727 fn rms(samples: &[f32]) -> f32 {
728 (samples.iter().map(|&s| s * s).sum::<f32>() / samples.len() as f32).sqrt()
729 }
730
731 /// Feed one block through the canceller: push the far-end reference,
732 /// process the mic block, return the output.
733 fn step(aec: &mut Aec, far_end: &AecFarEnd, mic_chunk: &[f32], far_chunk: &[f32]) -> Vec<f32> {
734 far_end.push_f32(far_chunk);
735 let mut samples = mic_chunk.to_vec();
736 let mut bus = AudioBus {
737 samples: &mut samples,
738 sample_rate: SR,
739 };
740 aec.process(&mut bus);
741 for &s in &samples {
742 debug_assert!(!s.is_nan(), "aec produced NaN output");
743 }
744 samples
745 }
746
747 #[test]
748 fn converges_on_synthetic_echo() {
749 let h = rir_taps();
750 let mut far_rng = Xorshift32::new(0x1234_5678);
751 // mu = 0.1 (the measured-safe default) converges ~5x slower than
752 // the old 0.5 — same ERLE floors, longer runway.
753 let total = 24 * SR as usize;
754 let far: Vec<f32> = (0..total).map(|_| 0.3 * far_rng.next_signed()).collect();
755 let clean = convolve_causal(&far, &h);
756 let mut noise_rng = Xorshift32::new(0x9e37_79b9);
757 let mic: Vec<f32> = clean
758 .iter()
759 .map(|&c| c + 1e-4 * noise_rng.next_signed())
760 .collect();
761
762 let (mut aec, far_end) = Aec::new(
763 AecConfig {
764 tail_ms: 128,
765 delay_ms: 0,
766 mu: 0.05,
767 },
768 SR,
769 );
770
771 let mut output = Vec::with_capacity(total);
772 for start in (0..total).step_by(BLOCK) {
773 let range = start..start + BLOCK;
774 let out = step(&mut aec, &far_end, &mic[range.clone()], &far[range]);
775 output.extend(out);
776 if start + BLOCK == 16 * SR as usize {
777 assert!(aec.erle_db() > 12.0, "erle at 16s = {}", aec.erle_db());
778 }
779 }
780
781 let last_1s = SR as usize;
782 let out_rms = rms(&output[output.len() - last_1s..]);
783 let mic_rms = rms(&mic[mic.len() - last_1s..]);
784 assert!(
785 out_rms < 0.25 * mic_rms,
786 "out_rms={out_rms} mic_rms={mic_rms} (want out_rms < 25% of mic_rms)"
787 );
788 }
789
790 #[test]
791 fn no_far_end_is_passthrough() {
792 let total = 2 * SR as usize;
793 let mut rng = Xorshift32::new(0xdead_beef);
794 let mic: Vec<f32> = (0..total)
795 .map(|i| {
796 let t = i as f32 / SR as f32;
797 0.2 * (2.0 * std::f32::consts::PI * 220.0 * t).sin() + 0.05 * rng.next_signed()
798 })
799 .collect();
800
801 let (mut aec, _far_end) = Aec::new(AecConfig::default(), SR);
802 let mut output = Vec::with_capacity(total);
803 for start in (0..total).step_by(BLOCK) {
804 let mut samples = mic[start..start + BLOCK].to_vec();
805 let mut bus = AudioBus {
806 samples: &mut samples,
807 sample_rate: SR,
808 };
809 aec.process(&mut bus);
810 output.extend(samples);
811 }
812
813 let latency = aec.latency_samples();
814 assert_eq!(latency, BLOCK);
815 assert!(output[..latency].iter().all(|&s| s == 0.0));
816 for i in latency..output.len() {
817 assert_eq!(output[i], mic[i - latency], "mismatch at sample {i}");
818 }
819 assert_eq!(aec.erle_db(), 0.0);
820 }
821
822 #[test]
823 fn double_talk_freezes_adaptation() {
824 let h = rir_taps();
825 let mut far_rng = Xorshift32::new(0x7777_8888);
826 // 14 s to converge (mu = 0.1) + 0.5 s double-talk burst + recovery.
827 let total = 20 * SR as usize;
828 let far: Vec<f32> = (0..total).map(|_| 0.3 * far_rng.next_signed()).collect();
829 let clean = convolve_causal(&far, &h);
830 let mut noise_rng = Xorshift32::new(0xaaaa_bbbb);
831 let mut mic: Vec<f32> = clean
832 .iter()
833 .map(|&c| c + 1e-4 * noise_rng.next_signed())
834 .collect();
835
836 let burst_start = 14 * SR as usize;
837 let burst_len = SR as usize / 2;
838 for (i, sample) in mic[burst_start..burst_start + burst_len]
839 .iter_mut()
840 .enumerate()
841 {
842 let t = i as f32 / SR as f32;
843 *sample += 0.8 * (2.0 * std::f32::consts::PI * 440.0 * t).sin();
844 }
845
846 let (mut aec, far_end) = Aec::new(
847 AecConfig {
848 tail_ms: 128,
849 delay_ms: 0,
850 mu: 0.05,
851 },
852 SR,
853 );
854
855 let mut burst_out = Vec::with_capacity(burst_len);
856 for start in (0..total).step_by(BLOCK) {
857 let range = start..start + BLOCK;
858 let out = step(&mut aec, &far_end, &mic[range.clone()], &far[range]);
859 if start == burst_start - BLOCK {
860 assert!(aec.erle_db() > 12.0, "pre-burst erle = {}", aec.erle_db());
861 }
862 if start >= burst_start && start < burst_start + burst_len {
863 burst_out.extend(out);
864 }
865 }
866
867 // (a) the near-end burst survives adaptation-frozen cancellation.
868 let expected_burst_rms = 0.8 / std::f32::consts::SQRT_2;
869 let burst_rms = rms(&burst_out);
870 assert!(
871 burst_rms > 0.7 * expected_burst_rms,
872 "burst_rms={burst_rms} expected>={}",
873 0.7 * expected_burst_rms
874 );
875
876 // (b) weights survived the burst untouched: ERLE recovers.
877 assert!(
878 aec.erle_db() > 10.0,
879 "post-recovery erle = {}",
880 aec.erle_db()
881 );
882 }
883
884 #[test]
885 fn delay_compensation_works() {
886 let h = rir_taps();
887 let delay_samples = 640; // 40 ms @ 16 kHz
888 let mut far_rng = Xorshift32::new(0x2222_3333);
889 // mu = 0.1 (the measured-safe default) converges ~5x slower than
890 // the old 0.5 — same ERLE floors, longer runway.
891 let total = 24 * SR as usize;
892 let far: Vec<f32> = (0..total).map(|_| 0.3 * far_rng.next_signed()).collect();
893
894 let mut far_delayed = vec![0.0f32; total];
895 far_delayed[delay_samples..].copy_from_slice(&far[..total - delay_samples]);
896 let clean = convolve_causal(&far_delayed, &h);
897 let mut noise_rng = Xorshift32::new(0x4444_5555);
898 let mic: Vec<f32> = clean
899 .iter()
900 .map(|&c| c + 1e-4 * noise_rng.next_signed())
901 .collect();
902
903 let (mut aec, far_end) = Aec::new(
904 AecConfig {
905 tail_ms: 128,
906 delay_ms: 40,
907 mu: 0.05,
908 },
909 SR,
910 );
911
912 for start in (0..total).step_by(BLOCK) {
913 let range = start..start + BLOCK;
914 step(&mut aec, &far_end, &mic[range.clone()], &far[range]);
915 }
916
917 assert!(aec.erle_db() > 12.0, "erle = {}", aec.erle_db());
918 }
919
920 #[test]
921 fn erle_reports_zero_without_activity() {
922 let (mut aec, _far_end) = Aec::new(AecConfig::default(), SR);
923 assert_eq!(aec.erle_db(), 0.0);
924
925 let mut samples = vec![0.01f32; BLOCK];
926 let mut bus = AudioBus {
927 samples: &mut samples,
928 sample_rate: SR,
929 };
930 aec.process(&mut bus);
931 assert_eq!(aec.erle_db(), 0.0);
932 }
933}