Module aec 

Acoustic echo cancellation: subtract the bot’s own voice from the mic before anything else touches it.

Why this stage exists

On an open-speaker device (phone speakerphone, conference room, a laptop with no headset) the bot’s own synthesized voice leaves the speaker and re-enters the microphone a few milliseconds to a few hundred milliseconds later, attenuated and reshaped by the room. Under client-interruption authority (the model treats any mic energy as “the user is talking, stop”) that echo makes the bot interrupt itself mid-sentence. The fix is not a bigger VAD threshold — it is removing the echo from the signal the VAD (and the ASR, and the model) ever sees.

The far-end reference — the audio actually handed to the speaker — is available in this SDK on the playback path. Aec::new returns an AecFarEnd handle; feed it the same PCM the speaker plays (AecFarEnd::push_pcm16 / AecFarEnd::push_f32) and this stage predicts and subtracts the echo before it reaches the rest of the mic chain. This stage must run before the denoiser — RNNoise (and any other nonlinear enhancer) rewrites the spectrum in ways that break the linear room-response model the adaptive filter is trying to learn; feed it a denoised signal and it converges on garbage, if it converges at all.

Algorithm: partitioned-block frequency-domain NLMS (overlap-save)

This is a linear echo canceller — it models the echo path as an FIR filter of tail_ms and adapts that filter with normalized LMS, done in the frequency domain per 10 ms block for efficiency (one FFT pair per block handles a filter tail hundreds of taps long). It does not handle nonlinear echo (cheap-speaker clipping/distortion) — that needs a nonlinear residual-echo suppressor layered after this stage, out of scope here.

• Block size B = 10 ms of samples (160 at 16 kHz). FFT size N = 2B (overlap-save discipline: a linear B-tap convolution result is only valid in the last B samples of a 2B-point circular convolution, so every inverse transform here keeps only that half and discards the first B as circular-wraparound garbage).
• The echo path is split into P = ceil(tail_ms / 10ms) partitions of B taps each, one weight vector W_p (complex, B+1 one-sided bins) per partition. Only one forward FFT of the far-end is computed per block; the P per-partition spectra are a rolling history of that same transform (a ring buffer), not P separate transforms — this is the entire point of “partitioned” convolution.
• Echo estimate: Y = Σ_p W_p ⊙ X_p; the time-domain estimate is the last B samples of IFFT(Y), scaled by 1/N (realfft/rustfft transforms are unnormalized in both directions, so a forward+inverse round trip scales amplitude by N unless corrected — see the source for where that correction lands).
• NLMS update in the frequency domain: W_p[k] += μ · conj(X_p[k]) · E[k] / (Px[k] + ε), where E = FFT([zeros(B), e]) (error zero-padded at the front — the adjoint of “keep only the last B samples” used to form the estimate) and Px[k] is an EWMA (0.9 retained / 0.1 new, the same convention DspChain uses for its meters) of Σ_p |X_p[k]|².
• Gradient constraint, applied every block: IFFT each just-updated W_p back to time domain, zero the last B samples (an unconstrained frequency-domain update can grow acausal/wraparound content that a real B-tap filter can’t have), FFT back. At this block size the cost (2P extra transforms/block) is cheap enough to just always pay it.

Double-talk protection

Adapting while the near-end user is also speaking teaches the filter to partially cancel the user, which is exactly backwards. Gating is cheap and Geigel-style: adaptation only runs when the far-end block power exceeds a floor (there is nothing to learn from if the bot is silent) and the mic peak for the block does not exceed 0.9 × the largest far-end peak seen in the last P blocks — a mic level comparable to (or louder than) the recent far-end implies near-end speech is present, since real echo return loss attenuates. A trip freezes adaptation (not echo cancellation — the existing filter keeps subtracting its prediction) for a ~30-block (~300 ms) hangover so a single loud consonant doesn’t cause the filter to start re-adapting mid-sentence.

Bulk delay

delay_ms (default 40) compensates the latency between “audio handed to AecFarEnd” and “that audio arrives at the mic via air/room path” — mostly playback buffering, not room propagation. It is implemented as a FIFO the far-end reference passes through before it ever reaches the filter. tail_ms must cover whatever misalignment remains after this coarse compensation (clock drift, a delay_ms that’s an estimate rather than measured, etc.) — the adaptive filter can only pull in echo that falls inside its P-partition window relative to the delayed reference; echo arriving earlier than the (delayed) reference cannot be modeled by a causal filter at all, and echo arriving later than tail_ms past it is simply not learned.

Known failure modes (stated, not hidden)

• Far-end underrun: if AecFarEnd hasn’t been fed enough audio to fill the delay line for a given mic block, the missing far-end samples are treated as silence. No echo is predicted for that block — the raw (uncancelled) mic audio passes through for whatever portion is missing.
• Startup: for the first delay_ms worth of audio the delay line is still draining its zero-fill primer, so there is nothing to cancel yet even if far-end audio is already flowing.
• This stage always adds exactly one block (B samples) of latency, declared honestly via DspStage::latency_samples — mic audio is buffered internally until a full block is available, processed, and the previous block’s result is what comes out, so arbitrary input chunk sizes are supported (a stage caller need not chunk to B itself) at the cost of that fixed one-block delay.
• Mono, single far-end source only. Stereo far-end / multiple simultaneous playback streams are out of scope.

Structs

Aec

Partitioned-block frequency-domain NLMS acoustic echo canceller. See the module docs for the algorithm and its stated limits.

AecConfig

Tunables for Aec::new. All three fields have the module’s tested defaults via Default.

AecFarEnd

Handle the playback side feeds with the same audio being sent to the speaker. Cheap to clone (shares one queue via Arc); safe to call from a different task/thread than the one driving Aec::process.