gemini_adk_fluent_rs/voice/dsp/
aec.rs

1//! Acoustic echo cancellation: subtract the bot's own voice from the mic
2//! before anything else touches it.
3//!
4//! # Why this stage exists
5//!
6//! On an open-speaker device (phone speakerphone, conference room, a laptop
7//! with no headset) the bot's own synthesized voice leaves the speaker and
8//! re-enters the microphone a few milliseconds to a few hundred milliseconds
9//! later, attenuated and reshaped by the room. Under client-interruption
10//! authority (the model treats any mic energy as "the user is talking, stop")
11//! that echo makes the bot interrupt itself mid-sentence. The fix is not a
12//! bigger VAD threshold — it is removing the echo from the signal the VAD
13//! (and the ASR, and the model) ever sees.
14//!
15//! The far-end reference — the audio actually handed to the speaker — is
16//! available in this SDK on the playback path. [`Aec::new`] returns an
17//! [`AecFarEnd`] handle; feed it the same PCM the speaker plays
18//! ([`AecFarEnd::push_pcm16`] / [`AecFarEnd::push_f32`]) and this stage
19//! predicts and subtracts the echo before it reaches the rest of the mic
20//! chain. **This stage must run before the denoiser** — RNNoise (and any
21//! other nonlinear enhancer) rewrites the spectrum in ways that break the
22//! linear room-response model the adaptive filter is trying to learn; feed
23//! it a denoised signal and it converges on garbage, if it converges at all.
24//!
25//! # Algorithm: partitioned-block frequency-domain NLMS (overlap-save)
26//!
27//! This is a linear echo canceller — it models the echo path as an FIR
28//! filter of `tail_ms` and adapts that filter with normalized LMS, done in
29//! the frequency domain per 10 ms block for efficiency (one FFT pair
30//! per block handles a filter tail hundreds of taps long). It does **not**
31//! handle nonlinear echo (cheap-speaker clipping/distortion) — that needs a
32//! nonlinear residual-echo suppressor layered after this stage, out of scope
33//! here.
34//!
35//! - Block size `B` = 10 ms of samples (160 at 16 kHz). FFT size
36//!   `N = 2B` (overlap-save discipline: a linear `B`-tap convolution result
37//!   is only valid in the *last* `B` samples of a `2B`-point circular
38//!   convolution, so every inverse transform here keeps only that half and
39//!   discards the first `B` as circular-wraparound garbage).
40//! - The echo path is split into `P = ceil(tail_ms / 10ms)` partitions of
41//!   `B` taps each, one weight vector `W_p` (complex, `B+1` one-sided bins)
42//!   per partition. Only **one** forward FFT of the far-end is computed per
43//!   block; the `P` per-partition spectra are a rolling history of that same
44//!   transform (a ring buffer), not `P` separate transforms — this is the
45//!   entire point of "partitioned" convolution.
46//! - Echo estimate: `Y = Σ_p W_p ⊙ X_p`; the time-domain estimate is the last
47//!   `B` samples of `IFFT(Y)`, scaled by `1/N` (`realfft`/`rustfft` transforms
48//!   are unnormalized in both directions, so a forward+inverse round trip
49//!   scales amplitude by `N` unless corrected — see the source for where
50//!   that correction lands).
51//! - NLMS update in the frequency domain: `W_p[k] += μ · conj(X_p[k]) ·
52//!   E[k] / (Px[k] + ε)`, where `E = FFT([zeros(B), e])` (error zero-padded
53//!   at the *front* — the adjoint of "keep only the last `B` samples" used
54//!   to form the estimate) and `Px[k]` is an EWMA (0.9 retained / 0.1 new,
55//!   the same convention [`DspChain`](super::DspChain) uses for its meters)
56//!   of `Σ_p |X_p[k]|²`.
57//! - Gradient constraint, applied every block: IFFT each just-updated `W_p`
58//!   back to time domain, zero the *last* `B` samples (an unconstrained
59//!   frequency-domain update can grow acausal/wraparound content that a
60//!   real `B`-tap filter can't have), FFT back. At this block size the cost
61//!   (`2P` extra transforms/block) is cheap enough to just always pay it.
62//!
63//! # Double-talk protection
64//!
65//! Adapting while the near-end user is also speaking teaches the filter to
66//! partially cancel the user, which is exactly backwards. Gating is cheap
67//! and Geigel-style: adaptation only runs when the far-end block power
68//! exceeds a floor (there is nothing to learn from if the bot is silent)
69//! *and* the mic peak for the block does not exceed `0.9 ×` the largest
70//! far-end peak seen in the last `P` blocks — a mic level comparable to (or
71//! louder than) the recent far-end implies near-end speech is present, since
72//! real echo return loss attenuates. A trip freezes adaptation (not echo
73//! cancellation — the existing filter keeps subtracting its prediction) for
74//! a ~30-block (~300 ms) hangover so a single loud consonant doesn't
75//! cause the filter to start re-adapting mid-sentence.
76//!
77//! # Bulk delay
78//!
79//! `delay_ms` (default 40) compensates the latency between "audio handed to
80//! [`AecFarEnd`]" and "that audio arrives at the mic via air/room path" —
81//! mostly playback buffering, not room propagation. It is implemented as a
82//! FIFO the far-end reference passes through before it ever reaches the
83//! filter. **`tail_ms` must cover whatever misalignment remains** after this
84//! coarse compensation (clock drift, a `delay_ms` that's an estimate rather
85//! than measured, etc.) — the adaptive filter can only pull in echo that
86//! falls inside its `P`-partition window relative to the delayed reference;
87//! echo arriving *earlier* than the (delayed) reference cannot be modeled by
88//! a causal filter at all, and echo arriving later than `tail_ms` past it
89//! is simply not learned.
90//!
91//! # Known failure modes (stated, not hidden)
92//!
93//! - **Far-end underrun**: if [`AecFarEnd`] hasn't been fed enough audio to
94//!   fill the delay line for a given mic block, the missing far-end samples
95//!   are treated as silence. No echo is predicted for that block — the raw
96//!   (uncancelled) mic audio passes through for whatever portion is missing.
97//! - **Startup**: for the first `delay_ms` worth of audio the delay line is
98//!   still draining its zero-fill primer, so there is nothing to cancel yet
99//!   even if far-end audio is already flowing.
100//! - **This stage always adds exactly one block (`B` samples) of latency**,
101//!   declared honestly via [`DspStage::latency_samples`] — mic audio is
102//!   buffered internally until a full block is available, processed, and
103//!   the *previous* block's result is what comes out, so arbitrary input
104//!   chunk sizes are supported (a stage caller need not chunk to `B` itself)
105//!   at the cost of that fixed one-block delay.
106//! - **Mono, single far-end source only.** Stereo far-end / multiple
107//!   simultaneous playback streams are out of scope.
108
109use std::collections::VecDeque;
110use std::sync::Arc;
111
112use parking_lot::Mutex;
113use realfft::num_complex::Complex32;
114use realfft::{ComplexToReal, RealFftPlanner, RealToComplex};
115
116use super::{AudioBus, DspStage};
117
118/// Far-end block power below this is "the bot isn't talking" — don't adapt,
119/// there is nothing to learn an echo path from.
120const FAR_ACTIVE_THRESHOLD: f32 = 1e-6;
121/// Geigel double-talk ratio: mic peak vs. the recent far-end peak.
122const GEIGEL_RATIO: f32 = 0.9;
123/// Blocks to keep adaptation frozen after a double-talk trip (~300 ms at
124/// the 10 ms block size).
125const HANGOVER_BLOCKS: u32 = 30;
126/// NLMS regularization floor — prevents division blow-up when the far-end
127/// spectrum is near zero. Negligible next to any real signal's power.
128const EPS: f32 = 1e-6;
129/// EWMA retain weight for the `Px` power estimate and the ERLE meters —
130/// matches the smoothing convention already used by
131/// [`DspChain`](super::DspChain)'s stage meters.
132const EWMA_RETAIN: f32 = 0.9;
133/// How much far-end audio [`AecFarEnd`] buffers before dropping the oldest
134/// samples — bounds memory if the far-end producer runs ahead of the mic
135/// consumer indefinitely.
136const FAR_QUEUE_SECONDS: usize = 2;
137
138/// Tunables for [`Aec::new`]. All three fields have the module's tested
139/// defaults via [`Default`].
140pub struct AecConfig {
141    /// Length of the modeled echo tail, in milliseconds. Rounded up to a
142    /// whole number of 10 ms partitions. Must cover the room's actual
143    /// reverberant echo plus any residual misalignment left after
144    /// `delay_ms` — see the module docs on bulk delay.
145    pub tail_ms: u32,
146    /// Bulk delay compensation applied to the far-end reference before it
147    /// enters the filter, in milliseconds — models playback pipeline
148    /// latency between "handed to the speaker" and "captured by the mic".
149    pub delay_ms: u32,
150    /// NLMS step size. The theoretical bound for a power-normalized
151    /// update is `0 < mu < 2`; the practical bound on narrowband bursty
152    /// far ends (speech through a speaker) measured far below it:
153    /// 0.25+ diverges within seconds, 0.1 reaches a bad marginal
154    /// equilibrium (gradient-noise misadjustment above the echo level),
155    /// 0.05 holds through every measured stress with ERLE +8..11 dB on
156    /// the worst-case harmonic proxy and 12+ dB on broadband far ends.
157    /// Default 0.05 — raise it only with `evals/dspbench` watching.
158    pub mu: f32,
159}
160
161impl Default for AecConfig {
162    fn default() -> Self {
163        Self {
164            tail_ms: 128,
165            delay_ms: 40,
166            mu: 0.05,
167        }
168    }
169}
170
171/// Handle the playback side feeds with the same audio being sent to the
172/// speaker. Cheap to clone (shares one queue via `Arc`); safe to call from
173/// a different task/thread than the one driving [`Aec::process`].
174#[derive(Clone)]
175pub struct AecFarEnd {
176    queue: Arc<Mutex<VecDeque<f32>>>,
177    max_len: usize,
178}
179
180impl AecFarEnd {
181    /// Push PCM16 samples (converted to `f32` by `/32768`), oldest dropped
182    /// first if the internal buffer is over its bound.
183    pub fn push_pcm16(&self, samples: &[i16]) {
184        let mut q = self.queue.lock();
185        for &s in samples {
186            push_bounded(&mut q, self.max_len, f32::from(s) / 32768.0);
187        }
188    }
189
190    /// Push `f32` samples directly (nominal `[-1.0, 1.0]`), oldest dropped
191    /// first if the internal buffer is over its bound.
192    pub fn push_f32(&self, samples: &[f32]) {
193        let mut q = self.queue.lock();
194        for &s in samples {
195            push_bounded(&mut q, self.max_len, s);
196        }
197    }
198}
199
200fn push_bounded(q: &mut VecDeque<f32>, max_len: usize, sample: f32) {
201    if q.len() >= max_len {
202        q.pop_front();
203    }
204    q.push_back(sample);
205}
206
207/// Partitioned-block frequency-domain NLMS acoustic echo canceller. See the
208/// module docs for the algorithm and its stated limits.
209pub struct Aec {
210    sample_rate: u32,
211    /// Block size `B`, in samples (10 ms).
212    block: usize,
213    /// FFT size `N = 2B`.
214    fft_len: usize,
215    /// Number of `B`-tap partitions covering `tail_ms`.
216    partitions: usize,
217    mu: f32,
218
219    r2c: Arc<dyn RealToComplex<f32>>,
220    c2r: Arc<dyn ComplexToReal<f32>>,
221    r2c_scratch: Vec<Complex32>,
222    c2r_scratch: Vec<Complex32>,
223    /// Reusable time-domain scratch, length `fft_len`. Contents are garbage
224    /// between steps within a block — never read without first being
225    /// written in the same block.
226    time_scratch: Vec<f32>,
227
228    /// Ring buffer of the far-end spectrum, one slot per partition; `head`
229    /// names the slot holding the newest (`X_0`) transform.
230    x_hist: Vec<Vec<Complex32>>,
231    head: usize,
232    /// Adaptive filter weights, one vector per logical partition (fixed
233    /// indexing — `w[p]` always means "coefficients for lag `p` blocks",
234    /// unlike `x_hist` which rotates).
235    w: Vec<Vec<Complex32>>,
236    /// EWMA of the far-end power spectrum, `Σ_p |X_p[k]|²`.
237    px: Vec<f32>,
238    /// Previous far-end block, for building the overlap-save `[prev, cur]`
239    /// window.
240    far_hist_prev: Vec<f32>,
241    y_freq: Vec<Complex32>,
242    e_freq: Vec<Complex32>,
243
244    mic_block: Vec<f32>,
245    far_block: Vec<f32>,
246    error: Vec<f32>,
247
248    in_fifo: VecDeque<f32>,
249    out_fifo: VecDeque<f32>,
250    far_queue: Arc<Mutex<VecDeque<f32>>>,
251    far_delay_buf: VecDeque<f32>,
252
253    far_peak_hist: VecDeque<f32>,
254    hangover: u32,
255
256    erle_ever_active: bool,
257    erle_mic_pow: f32,
258    erle_err_pow: f32,
259}
260
261impl Aec {
262    /// Build the canceller and its paired far-end feed handle. `sample_rate`
263    /// must match the [`AudioBus`] this stage will be run on.
264    pub fn new(config: AecConfig, sample_rate: u32) -> (Self, AecFarEnd) {
265        let block = ((u64::from(sample_rate) * 10 / 1000) as usize).max(1);
266        let fft_len = block * 2;
267        let bins = block + 1;
268
269        let tail_samples = u64::from(config.tail_ms) * u64::from(sample_rate) / 1000;
270        let partitions = ((tail_samples as f64 / block as f64).ceil() as usize).max(1);
271        let delay_samples = (u64::from(config.delay_ms) * u64::from(sample_rate) / 1000) as usize;
272
273        let mut planner = RealFftPlanner::<f32>::new();
274        let r2c = planner.plan_fft_forward(fft_len);
275        let c2r = planner.plan_fft_inverse(fft_len);
276        let r2c_scratch = r2c.make_scratch_vec();
277        let c2r_scratch = c2r.make_scratch_vec();
278
279        let zero_bins = || vec![Complex32::new(0.0, 0.0); bins];
280        let x_hist = (0..partitions).map(|_| zero_bins()).collect();
281        let w = (0..partitions).map(|_| zero_bins()).collect();
282
283        let mut out_fifo = VecDeque::with_capacity(block * 2);
284        out_fifo.extend(std::iter::repeat_n(0.0f32, block));
285
286        let mut far_delay_buf = VecDeque::with_capacity(delay_samples + block);
287        far_delay_buf.extend(std::iter::repeat_n(0.0f32, delay_samples));
288
289        let far_queue = Arc::new(Mutex::new(VecDeque::new()));
290        let max_len = sample_rate as usize * FAR_QUEUE_SECONDS;
291
292        let aec = Self {
293            sample_rate,
294            block,
295            fft_len,
296            partitions,
297            mu: config.mu,
298            r2c,
299            c2r,
300            r2c_scratch,
301            c2r_scratch,
302            time_scratch: vec![0.0; fft_len],
303            x_hist,
304            head: 0,
305            w,
306            px: vec![0.0; bins],
307            far_hist_prev: vec![0.0; block],
308            y_freq: zero_bins(),
309            e_freq: zero_bins(),
310            mic_block: Vec::with_capacity(block),
311            far_block: Vec::with_capacity(block),
312            error: Vec::with_capacity(block),
313            in_fifo: VecDeque::new(),
314            out_fifo,
315            far_queue: Arc::clone(&far_queue),
316            far_delay_buf,
317            far_peak_hist: VecDeque::with_capacity(partitions),
318            hangover: 0,
319            erle_ever_active: false,
320            erle_mic_pow: 0.0,
321            erle_err_pow: 0.0,
322        };
323        (
324            aec,
325            AecFarEnd {
326                queue: far_queue,
327                max_len,
328            },
329        )
330    }
331
332    /// Echo Return Loss Enhancement, in dB: `10·log10(EWMA(mic power) /
333    /// EWMA(error power))`, measured only over blocks where the far-end was
334    /// active. `0.0` before any far-end activity has ever been observed —
335    /// there is nothing yet to report a ratio over.
336    pub fn erle_db(&self) -> f32 {
337        if !self.erle_ever_active {
338            return 0.0;
339        }
340        10.0 * (self.erle_mic_pow.max(1e-20) / self.erle_err_pow.max(1e-20)).log10()
341    }
342
343    fn process_block(&mut self) {
344        let block = self.block;
345        let n = self.fft_len;
346        let r2c = Arc::clone(&self.r2c);
347        let c2r = Arc::clone(&self.c2r);
348
349        // 1. Pull one block of mic samples and the delayed far-end reference.
350        self.mic_block.clear();
351        self.mic_block.extend(self.in_fifo.drain(0..block));
352
353        {
354            let mut q = self.far_queue.lock();
355            self.far_delay_buf.extend(q.drain(..));
356        }
357        let avail = self.far_delay_buf.len().min(block);
358        self.far_block.clear();
359        self.far_block.extend(self.far_delay_buf.drain(0..avail));
360        // Underrun: missing far-end samples are silence (no echo predicted
361        // for them) — see the module docs' failure-modes section.
362        self.far_block.resize(block, 0.0);
363
364        // 2. Overlap-save window [prev B, cur B] -> one forward FFT, stored
365        // as the newest ring slot (X_0).
366        self.time_scratch[..block].copy_from_slice(&self.far_hist_prev);
367        self.time_scratch[block..].copy_from_slice(&self.far_block);
368        self.head = (self.head + 1) % self.partitions;
369        r2c.process_with_scratch(
370            &mut self.time_scratch,
371            &mut self.x_hist[self.head],
372            &mut self.r2c_scratch,
373        )
374        .expect("aec: far-end forward fft (fixed sizes)");
375        self.far_hist_prev.copy_from_slice(&self.far_block);
376
377        // 3. Echo estimate Y = sum_p W_p * X_p; keep the last B samples of
378        // IFFT(Y)/N (overlap-save: the first B are circular garbage).
379        for c in &mut self.y_freq {
380            *c = Complex32::new(0.0, 0.0);
381        }
382        for p in 0..self.partitions {
383            let idx = (self.head + self.partitions - p) % self.partitions;
384            let xp = &self.x_hist[idx];
385            let wp = &self.w[p];
386            for ((y, w), x) in self.y_freq.iter_mut().zip(wp.iter()).zip(xp.iter()) {
387                *y += *w * *x;
388            }
389        }
390        c2r.process_with_scratch(
391            &mut self.y_freq,
392            &mut self.time_scratch,
393            &mut self.c2r_scratch,
394        )
395        .expect("aec: echo inverse fft (fixed sizes)");
396        let norm = 1.0 / n as f32;
397
398        // 4. Error = mic - echo estimate. This is the stage's output.
399        let echo_tail = &self.time_scratch[block..];
400        self.error.clear();
401        self.error.extend(
402            self.mic_block
403                .iter()
404                .zip(echo_tail.iter())
405                .map(|(&mic, &y)| mic - y * norm),
406        );
407        let mic_pow = mean_power(&self.mic_block);
408        let err_pow = mean_power(&self.error);
409
410        // 5. Error spectrum for the gradient: E = FFT([zeros(B), e]) — the
411        // zero-padding-at-the-front is the adjoint of step 3's "keep only
412        // the last B samples".
413        self.time_scratch[..block].fill(0.0);
414        self.time_scratch[block..].copy_from_slice(&self.error);
415        r2c.process_with_scratch(
416            &mut self.time_scratch,
417            &mut self.e_freq,
418            &mut self.r2c_scratch,
419        )
420        .expect("aec: error forward fft (fixed sizes)");
421
422        // 6. Px EWMA of the far-end power spectrum (NLMS normalizer).
423        // Held frozen while the far end is silent: decaying it through
424        // inter-burst gaps would make the next burst onset see a fraction
425        // of the true power and overshoot the NLMS stability bound —
426        // measured as divergence on speech-like (bursty) far ends.
427        let far_active = mean_power(&self.far_block) > FAR_ACTIVE_THRESHOLD;
428        if far_active {
429            for px in &mut self.px {
430                *px *= EWMA_RETAIN;
431            }
432            for p in 0..self.partitions {
433                let idx = (self.head + self.partitions - p) % self.partitions;
434                let xp = &self.x_hist[idx];
435                for (px, x) in self.px.iter_mut().zip(xp.iter()) {
436                    *px += (1.0 - EWMA_RETAIN) * x.norm_sqr();
437                }
438            }
439        }
440
441        // 7. Far-end-activity + Geigel double-talk gating.
442        let far_peak = self.far_block.iter().fold(0.0f32, |m, &s| m.max(s.abs()));
443        let mic_peak = self.mic_block.iter().fold(0.0f32, |m, &s| m.max(s.abs()));
444
445        if self.far_peak_hist.len() == self.partitions {
446            self.far_peak_hist.pop_front();
447        }
448        self.far_peak_hist.push_back(far_peak);
449        let max_far_peak = self.far_peak_hist.iter().fold(0.0f32, |m, &s| m.max(s));
450
451        let double_talk = far_active && mic_peak > GEIGEL_RATIO * max_far_peak;
452        if double_talk {
453            self.hangover = HANGOVER_BLOCKS;
454        }
455        let adapting = far_active && self.hangover == 0;
456        if self.hangover > 0 {
457            self.hangover -= 1;
458        }
459
460        // 8. NLMS update, gated by double-talk/far-end-activity.
461        if adapting {
462            let mu = self.mu;
463            // Leaky NLMS: damp weight components the far end never
464            // excites. Narrowband input leaves most bins unexcited; energy
465            // accumulating there from error noise is what turned marginal
466            // stability into a bad equilibrium (output above mic) on
467            // sustained harmonic far ends. ~0.1% decay per 10 ms block is
468            // invisible to converged echo paths (they are re-excited every
469            // block) and, together with the unexcited-bin gate below, fatal
470            // to the parasitic modes. (0.1%/block measured too strong: it
471            // capped converged broadband ERLE at ~9 dB.)
472            const LEAK: f32 = 0.9999;
473            for wp in &mut self.w {
474                for w in wp.iter_mut() {
475                    *w *= LEAK;
476                }
477            }
478            // Spectral regularization: floor every bin's normalizer at 1%
479            // of the mean bin power. In bins where a narrowband far end
480            // has no energy, Px is microscopic and the update would divide
481            // by EPS alone — the weights there random-walk on error noise
482            // until feedback diverges (measured on harmonic far ends).
483            let px_mean = self.px.iter().sum::<f32>() / self.px.len().max(1) as f32;
484            let px_floor = px_mean * 0.01;
485            for p in 0..self.partitions {
486                let idx = (self.head + self.partitions - p) % self.partitions;
487                let xp = &self.x_hist[idx];
488                let wp = &mut self.w[p];
489                let updates = wp
490                    .iter_mut()
491                    .zip(xp.iter())
492                    .zip(self.px.iter())
493                    .zip(self.e_freq.iter());
494                for (((w, x), px), e) in updates {
495                    // Bins the far end doesn't excite get NO update at all
496                    // (their weights only leak toward zero): with narrowband
497                    // input, updating unexcited bins lets error noise
498                    // accumulate parasitic weight energy that ends up
499                    // louder than the echo it was meant to cancel.
500                    if *px < px_floor {
501                        continue;
502                    }
503                    // Floor the normalizer with this bin's instantaneous
504                    // power: whatever the EWMA lags, the effective step
505                    // stays <= mu, inside the NLMS stability bound.
506                    let denom = px.max(x.norm_sqr()) + EPS;
507                    let grad = x.conj() * *e;
508                    let mut updated = *w + grad * (mu / denom);
509                    if !updated.re.is_finite() || !updated.im.is_finite() {
510                        updated = Complex32::new(0.0, 0.0);
511                    }
512                    *w = updated;
513                }
514            }
515        }
516
517        // 9. Gradient constraint (every block): project each partition back
518        // onto a causal B-tap filter so an unconstrained frequency-domain
519        // step can't grow acausal/wraparound content.
520        for p in 0..self.partitions {
521            c2r.process_with_scratch(
522                &mut self.w[p],
523                &mut self.time_scratch,
524                &mut self.c2r_scratch,
525            )
526            .expect("aec: constraint inverse fft (fixed sizes)");
527            for s in &mut self.time_scratch {
528                *s *= norm;
529            }
530            for s in &mut self.time_scratch[block..] {
531                *s = 0.0;
532            }
533            r2c.process_with_scratch(
534                &mut self.time_scratch,
535                &mut self.w[p],
536                &mut self.r2c_scratch,
537            )
538            .expect("aec: constraint forward fft (fixed sizes)");
539        }
540
541        // 10. ERLE bookkeeping — only on blocks where cancellation is
542        // measurable: far end active AND no double-talk (during double
543        // talk the error carries near-end speech by design; feeding those
544        // blocks poisons the meter for many seconds afterwards).
545        if adapting {
546            // Slow meter (~2 s memory): the fast Px constant would let
547            // burst onsets and AM valleys drag the reading far below the
548            // converged cancellation the waveforms show.
549            const ERLE_RETAIN: f32 = 0.995;
550            if self.erle_ever_active {
551                self.erle_mic_pow = self.erle_mic_pow * ERLE_RETAIN + mic_pow * (1.0 - ERLE_RETAIN);
552                self.erle_err_pow = self.erle_err_pow * ERLE_RETAIN + err_pow * (1.0 - ERLE_RETAIN);
553            } else {
554                self.erle_mic_pow = mic_pow;
555                self.erle_err_pow = err_pow;
556                self.erle_ever_active = true;
557            }
558        }
559        self.out_fifo.extend(self.error.iter().copied());
560    }
561}
562
563fn mean_power(samples: &[f32]) -> f32 {
564    if samples.is_empty() {
565        return 0.0;
566    }
567    let energy: f64 = samples.iter().map(|&s| f64::from(s) * f64::from(s)).sum();
568    (energy / samples.len() as f64) as f32
569}
570
571impl DspStage for Aec {
572    fn name(&self) -> &'static str {
573        "aec"
574    }
575
576    fn process(&mut self, bus: &mut AudioBus) {
577        debug_assert_eq!(
578            bus.sample_rate, self.sample_rate,
579            "Aec was built for a different sample rate than the bus it's running on"
580        );
581        let want = bus.samples.len();
582        self.in_fifo.extend(bus.samples.iter().copied());
583        while self.in_fifo.len() >= self.block {
584            self.process_block();
585        }
586        debug_assert!(self.out_fifo.len() >= want, "aec output FIFO underrun");
587        bus.samples.clear();
588        bus.samples.extend(self.out_fifo.drain(0..want));
589    }
590
591    fn latency_samples(&self) -> usize {
592        self.block
593    }
594}
595
596#[cfg(test)]
597mod tests {
598    #[test]
599    fn stays_stable_on_narrowband_bursty_far_end() {
600        // The discriminating input the white-noise tests miss: a harmonic,
601        // bursty far end (speech through a speaker). Three separate
602        // instabilities were measured here before their fixes: burst-onset
603        // step overshoot (decaying Px), empty-bin amplification (Px ~ 0 in
604        // bins the tone never touches), and a step size above the
605        // practical narrowband bound.
606        let sr = 16_000u32;
607        let n = 20 * sr as usize;
608        let mut far = vec![0.0f32; n];
609        let (on, off) = (sr as usize * 16 / 10, sr as usize * 6 / 10);
610        let mut i = 0usize;
611        let mut f0 = 120.0f32;
612        while i < n {
613            let end = (i + on).min(n);
614            for (k, slot) in far[i..end].iter_mut().enumerate() {
615                let t = k as f32 / sr as f32;
616                let am = 0.6 + 0.4 * (2.0 * std::f32::consts::PI * 4.0 * t).sin();
617                let mut v = 0.0;
618                for h in 1..=4 {
619                    v += (2.0 * std::f32::consts::PI * f0 * h as f32 * t).sin() / h as f32;
620                }
621                *slot = 0.12 * am * v;
622            }
623            f0 = if f0 > 190.0 { 120.0 } else { f0 + 23.0 };
624            i = end + off;
625        }
626        // Echo path: sparse decaying taps, 40 ms bulk delay.
627        let taps: [f32; 6] = [0.12, -0.05, 0.03, -0.015, 0.008, -0.004];
628        let delay = 40 * sr as usize / 1000;
629        let mut mic = vec![0.0f32; n];
630        for (j, m) in mic.iter_mut().enumerate() {
631            for (ti, &tap) in taps.iter().enumerate() {
632                let src = j as isize - delay as isize - (ti as isize * 37);
633                if src >= 0 {
634                    *m += tap * far[src as usize];
635                }
636            }
637        }
638
639        let (mut aec, far_end) = Aec::new(
640            AecConfig {
641                delay_ms: 40,
642                ..AecConfig::default()
643            },
644            sr,
645        );
646        let mut out = Vec::with_capacity(n);
647        for (idx, block) in mic.chunks(320).enumerate() {
648            let a = idx * 320;
649            far_end.push_f32(&far[a..(a + block.len()).min(n)]);
650            let mut buf = block.to_vec();
651            let mut bus = AudioBus {
652                samples: &mut buf,
653                sample_rate: sr,
654            };
655            aec.process(&mut bus);
656            out.extend_from_slice(&buf);
657        }
658
659        assert!(out.iter().all(|s| s.is_finite()), "output diverged");
660        let tail = &out[n - 2 * sr as usize..];
661        let mic_tail = &mic[n - 2 * sr as usize..];
662        let rms = |x: &[f32]| (x.iter().map(|s| s * s).sum::<f32>() / x.len() as f32).sqrt();
663        assert!(
664            rms(tail) < 0.5 * rms(mic_tail),
665            "no cancellation: out {} vs mic {}",
666            rms(tail),
667            rms(mic_tail)
668        );
669        assert!(aec.erle_db() > 6.0, "erle {} dB", aec.erle_db());
670    }
671
672    use super::*;
673
674    /// Fixed-seed xorshift32 — deterministic, no `rand` dependency.
675    struct Xorshift32(u32);
676
677    impl Xorshift32 {
678        fn new(seed: u32) -> Self {
679            Self(seed.max(1))
680        }
681
682        fn next_u32(&mut self) -> u32 {
683            let mut x = self.0;
684            x ^= x << 13;
685            x ^= x >> 17;
686            x ^= x << 5;
687            self.0 = x;
688            x
689        }
690
691        /// Uniform in `[-1.0, 1.0)`.
692        fn next_signed(&mut self) -> f32 {
693            (self.next_u32() as f32 / u32::MAX as f32).mul_add(2.0, -1.0)
694        }
695    }
696
697    const SR: u32 = 16_000;
698    const BLOCK: usize = 160;
699
700    /// 48-tap synthetic room response: decaying, sign-alternating every 4
701    /// taps (`0.05 * (-0.7)^(i/4)`). Scaled to `0.05` rather than the naive
702    /// `0.5` first-tap gain: repeating each decay step over 4 taps before
703    /// attenuating means the *sum* of the naive-amplitude taps exceeds unity
704    /// gain (a "louder than the far end" echo), which is not physically
705    /// realistic (echo return loss attenuates) and — more importantly for
706    /// the test — would spuriously trip the Geigel double-talk detector on
707    /// pure echo with no near-end speech at all.
708    fn rir_taps() -> Vec<f32> {
709        (0..48).map(|i: i32| 0.05 * (-0.7f32).powi(i / 4)).collect()
710    }
711
712    fn convolve_causal(far: &[f32], h: &[f32]) -> Vec<f32> {
713        let mut out = vec![0.0f32; far.len()];
714        for (n, out_n) in out.iter_mut().enumerate() {
715            let mut acc = 0.0f32;
716            for (i, &hi) in h.iter().enumerate() {
717                if i > n {
718                    break;
719                }
720                acc += hi * far[n - i];
721            }
722            *out_n = acc;
723        }
724        out
725    }
726
727    fn rms(samples: &[f32]) -> f32 {
728        (samples.iter().map(|&s| s * s).sum::<f32>() / samples.len() as f32).sqrt()
729    }
730
731    /// Feed one block through the canceller: push the far-end reference,
732    /// process the mic block, return the output.
733    fn step(aec: &mut Aec, far_end: &AecFarEnd, mic_chunk: &[f32], far_chunk: &[f32]) -> Vec<f32> {
734        far_end.push_f32(far_chunk);
735        let mut samples = mic_chunk.to_vec();
736        let mut bus = AudioBus {
737            samples: &mut samples,
738            sample_rate: SR,
739        };
740        aec.process(&mut bus);
741        for &s in &samples {
742            debug_assert!(!s.is_nan(), "aec produced NaN output");
743        }
744        samples
745    }
746
747    #[test]
748    fn converges_on_synthetic_echo() {
749        let h = rir_taps();
750        let mut far_rng = Xorshift32::new(0x1234_5678);
751        // mu = 0.1 (the measured-safe default) converges ~5x slower than
752        // the old 0.5 — same ERLE floors, longer runway.
753        let total = 24 * SR as usize;
754        let far: Vec<f32> = (0..total).map(|_| 0.3 * far_rng.next_signed()).collect();
755        let clean = convolve_causal(&far, &h);
756        let mut noise_rng = Xorshift32::new(0x9e37_79b9);
757        let mic: Vec<f32> = clean
758            .iter()
759            .map(|&c| c + 1e-4 * noise_rng.next_signed())
760            .collect();
761
762        let (mut aec, far_end) = Aec::new(
763            AecConfig {
764                tail_ms: 128,
765                delay_ms: 0,
766                mu: 0.05,
767            },
768            SR,
769        );
770
771        let mut output = Vec::with_capacity(total);
772        for start in (0..total).step_by(BLOCK) {
773            let range = start..start + BLOCK;
774            let out = step(&mut aec, &far_end, &mic[range.clone()], &far[range]);
775            output.extend(out);
776            if start + BLOCK == 16 * SR as usize {
777                assert!(aec.erle_db() > 12.0, "erle at 16s = {}", aec.erle_db());
778            }
779        }
780
781        let last_1s = SR as usize;
782        let out_rms = rms(&output[output.len() - last_1s..]);
783        let mic_rms = rms(&mic[mic.len() - last_1s..]);
784        assert!(
785            out_rms < 0.25 * mic_rms,
786            "out_rms={out_rms} mic_rms={mic_rms} (want out_rms < 25% of mic_rms)"
787        );
788    }
789
790    #[test]
791    fn no_far_end_is_passthrough() {
792        let total = 2 * SR as usize;
793        let mut rng = Xorshift32::new(0xdead_beef);
794        let mic: Vec<f32> = (0..total)
795            .map(|i| {
796                let t = i as f32 / SR as f32;
797                0.2 * (2.0 * std::f32::consts::PI * 220.0 * t).sin() + 0.05 * rng.next_signed()
798            })
799            .collect();
800
801        let (mut aec, _far_end) = Aec::new(AecConfig::default(), SR);
802        let mut output = Vec::with_capacity(total);
803        for start in (0..total).step_by(BLOCK) {
804            let mut samples = mic[start..start + BLOCK].to_vec();
805            let mut bus = AudioBus {
806                samples: &mut samples,
807                sample_rate: SR,
808            };
809            aec.process(&mut bus);
810            output.extend(samples);
811        }
812
813        let latency = aec.latency_samples();
814        assert_eq!(latency, BLOCK);
815        assert!(output[..latency].iter().all(|&s| s == 0.0));
816        for i in latency..output.len() {
817            assert_eq!(output[i], mic[i - latency], "mismatch at sample {i}");
818        }
819        assert_eq!(aec.erle_db(), 0.0);
820    }
821
822    #[test]
823    fn double_talk_freezes_adaptation() {
824        let h = rir_taps();
825        let mut far_rng = Xorshift32::new(0x7777_8888);
826        // 14 s to converge (mu = 0.1) + 0.5 s double-talk burst + recovery.
827        let total = 20 * SR as usize;
828        let far: Vec<f32> = (0..total).map(|_| 0.3 * far_rng.next_signed()).collect();
829        let clean = convolve_causal(&far, &h);
830        let mut noise_rng = Xorshift32::new(0xaaaa_bbbb);
831        let mut mic: Vec<f32> = clean
832            .iter()
833            .map(|&c| c + 1e-4 * noise_rng.next_signed())
834            .collect();
835
836        let burst_start = 14 * SR as usize;
837        let burst_len = SR as usize / 2;
838        for (i, sample) in mic[burst_start..burst_start + burst_len]
839            .iter_mut()
840            .enumerate()
841        {
842            let t = i as f32 / SR as f32;
843            *sample += 0.8 * (2.0 * std::f32::consts::PI * 440.0 * t).sin();
844        }
845
846        let (mut aec, far_end) = Aec::new(
847            AecConfig {
848                tail_ms: 128,
849                delay_ms: 0,
850                mu: 0.05,
851            },
852            SR,
853        );
854
855        let mut burst_out = Vec::with_capacity(burst_len);
856        for start in (0..total).step_by(BLOCK) {
857            let range = start..start + BLOCK;
858            let out = step(&mut aec, &far_end, &mic[range.clone()], &far[range]);
859            if start == burst_start - BLOCK {
860                assert!(aec.erle_db() > 12.0, "pre-burst erle = {}", aec.erle_db());
861            }
862            if start >= burst_start && start < burst_start + burst_len {
863                burst_out.extend(out);
864            }
865        }
866
867        // (a) the near-end burst survives adaptation-frozen cancellation.
868        let expected_burst_rms = 0.8 / std::f32::consts::SQRT_2;
869        let burst_rms = rms(&burst_out);
870        assert!(
871            burst_rms > 0.7 * expected_burst_rms,
872            "burst_rms={burst_rms} expected>={}",
873            0.7 * expected_burst_rms
874        );
875
876        // (b) weights survived the burst untouched: ERLE recovers.
877        assert!(
878            aec.erle_db() > 10.0,
879            "post-recovery erle = {}",
880            aec.erle_db()
881        );
882    }
883
884    #[test]
885    fn delay_compensation_works() {
886        let h = rir_taps();
887        let delay_samples = 640; // 40 ms @ 16 kHz
888        let mut far_rng = Xorshift32::new(0x2222_3333);
889        // mu = 0.1 (the measured-safe default) converges ~5x slower than
890        // the old 0.5 — same ERLE floors, longer runway.
891        let total = 24 * SR as usize;
892        let far: Vec<f32> = (0..total).map(|_| 0.3 * far_rng.next_signed()).collect();
893
894        let mut far_delayed = vec![0.0f32; total];
895        far_delayed[delay_samples..].copy_from_slice(&far[..total - delay_samples]);
896        let clean = convolve_causal(&far_delayed, &h);
897        let mut noise_rng = Xorshift32::new(0x4444_5555);
898        let mic: Vec<f32> = clean
899            .iter()
900            .map(|&c| c + 1e-4 * noise_rng.next_signed())
901            .collect();
902
903        let (mut aec, far_end) = Aec::new(
904            AecConfig {
905                tail_ms: 128,
906                delay_ms: 40,
907                mu: 0.05,
908            },
909            SR,
910        );
911
912        for start in (0..total).step_by(BLOCK) {
913            let range = start..start + BLOCK;
914            step(&mut aec, &far_end, &mic[range.clone()], &far[range]);
915        }
916
917        assert!(aec.erle_db() > 12.0, "erle = {}", aec.erle_db());
918    }
919
920    #[test]
921    fn erle_reports_zero_without_activity() {
922        let (mut aec, _far_end) = Aec::new(AecConfig::default(), SR);
923        assert_eq!(aec.erle_db(), 0.0);
924
925        let mut samples = vec![0.01f32; BLOCK];
926        let mut bus = AudioBus {
927            samples: &mut samples,
928            sample_rate: SR,
929        };
930        aec.process(&mut bus);
931        assert_eq!(aec.erle_db(), 0.0);
932    }
933}