What an AI voice enhancer actually does
The phrase covers two machines that share nothing. One subtracts noise from the recording you made. The other rebuilds your speech from scratch and hands you a waveform that was never recorded. Knowing which one you are pointing at a file answers every other question about it — including whether it can save a muffled mic, and whether you are allowed to quote the result. Our voice enhancer does both, on your own device, and makes you choose between them by name.
A filter can only take away
A denoiser looks at the spectrum of your recording and predicts a mask: for every frequency at every instant, how much of this is voice and how much is room. Multiply, invert the transform, done. The strong version of that idea — DeepFilterNet 3, which our background noise remover runs — is genuinely very good, works on a phone, and every word it returns is a word you said.
What it cannot do is add. If the microphone was under a coat, the high frequencies that make a consonant crisp were never captured, and there is no mask that multiplies them back into existence. If you recorded in a stairwell, the reverb is your own voice arriving late — not a separate sound sitting alongside it, so there is nothing for the mask to remove without removing you. Filters fail on muffling and echo for a structural reason, not for want of a better filter.
A rebuild throws your waveform away
The generative approach — resemble-enhance (MIT), from Resemble AI, is the open model our tool uses — never edits your audio. It runs five models over each two and a half seconds:
- 1. Denoiser (UNet). Masks the noise out, exactly like the filter above. Its output is not what you download — it is used as a hint for what follows.
- 2. Encoder (IRMAE). Turns a 128-band mel spectrogram of your original audio into a 64-dimensional latent — 128 numbers a frame compressed to 64, which is the space the next step actually works in.
- 3. Flow matching (CFM). Starts from that sketch mixed with noise and nudges it, step by step, towards what clean speech looks like — steered by the denoised spectrogram. Sixty-four evaluations per chunk. This is the step that invents.
- 4. Decoder (IRMAE). Expands the solved latent back into a 160-channel spectral feature.
- 5. Vocoder (UnivNet). Synthesises 44.1 kHz audio from those features and a noise seed. Nothing here is a copy of your recording; the samples are generated.
Chunks are cross-faded back together, and a one-minute memo costs roughly two thousand model invocations. That is why it can fix a muffled mic: it is not repairing your consonant, it is producing a new one from a model that knows what your consonant should sound like.
Which one you want
| Filter (denoise) | Rebuild (enhance) | |
|---|---|---|
| What comes out | Your recording, minus some of it. Every sample descends from a sample you recorded. | A new waveform, synthesised from a description of your speech. Nothing survives. |
| Can add detail that was never captured | No. Subtraction cannot restore a frequency the microphone never heard. | Yes — that is the point, and also the risk. |
| Fixes hiss, hum, fans, traffic | Yes, and this is the right tool for it. | Yes, but at a thousand times the compute. |
| Fixes a muffled mic or a hard room echo | No. Neither is additive noise. | Often. |
| Safe to quote as what was said | Yes. | No. |
What it costs on your own machine
Nothing about a rebuild requires a server, and ours does not use one. The five graphs are about 1.3 GB of one-time download, kept on the device afterwards; the run needs roughly 6 GB of free graphics memory at once, which an 8 GB card manages only with nothing else heavy open. Throughput measured on an RTX 2080 is 0.77x realtime — a two-minute recording took about a minute and a half. On the CPU backend the same chain measures 6.32x realtime — over six times the recording's length — so it is refused there rather than offered as a slow path.
The filter is a different order of cost entirely: one 41 MB model, no graphics card, a fraction of the recording's length. If your recording is basically fine, that is the whole job.
Where a rebuild lies
A reconstruction is not a recording, and the difference is not academic. The artefacts cluster in three places: sibilance, where an invented /s/ can whistle or smear; breaths and mouth noise, which the model may render as speech or drop entirely; and very quiet speech, where there was little for the encoder to describe and the flow solver fills the gap with something plausible. Long silences can also acquire sounds that were never in the room.
So: never use a rebuild as evidence of what was said. For an interview you will quote, a medical dictation, a legal note or anything a dispute could turn on, denoise it — the waveform stays yours. Keep the rebuild for the memo you want to be listenable. Any tool worth using hands you both files and lets you hear the difference before you choose; ours plays them side by side before you download either.
Try it on your own recording
The voice enhancer runs both modes in this browser tab — Denoise anywhere, Enhance on a GPU. Your files are never uploaded — everything runs on your device. If you only need the hiss gone from a video's audio track, the noise remover is the lighter tool; if you want the words rather than the audio, transcribe it.
Questions people actually ask
What does an AI voice enhancer actually do?
Two completely different things, sold under one phrase. A denoiser predicts which parts of your recording are noise and removes them — the words that come back are the words you said, in the recording you made. A generative enhancer converts your speech into a compact description (a mel spectrogram, then a compact numerical sketch of the speech), refines that description with a model trained on clean speech, and renders new audio from it with a neural vocoder. The second one is not a stronger version of the first; it is a different machine with a different failure mode.
Can it fix a muffled or echoey recording?
A filter cannot, and no amount of it will. Muffling is missing high frequencies — you cannot subtract your way to detail that the microphone never captured — and a room echo is your own voice convolved with the room, so a noise mask has nothing separate to remove. A generative rebuild can, because it is not editing your recording: it hears what you said, and renders that speech again as if the room and the microphone had been better. That is the one job that genuinely needs the heavier tool.
Does it change my words or my voice?
It is trained not to, and in practice keeps both. But the output is a reconstruction, not a recording, so it is not evidence of what was said. Sibilance, breaths and the quietest speech are where a rebuild differs audibly from the original, and long silences can pick up artefacts that were never in the room. If the recording itself matters — an interview you will quote, a medical or legal note — denoise it and leave the waveform alone.
Why does a rebuild need a graphics card when a denoiser does not?
Arithmetic. The denoiser is one pass over a ten-second window, and runs faster than real time on an ordinary processor. The generative chain runs an encoder, sixty-four solver evaluations, a decoder and a vocoder on every two and a half seconds of audio — about two thousand model invocations for a one-minute memo. Measured on the WASM backend that is 6.32x realtime, which is not a slow tool but an unusable one, so it is offered only where there is a GPU to run it.
Can this run without uploading my recording?
Yes, and it is what our tool does: five ONNX graphs downloaded once, about 1.3 GB in total, then executed in the browser tab through WebGPU. Nothing about the method requires a server — it requires memory and a capable graphics card — which many desktops and gaming laptops have, and phones do not. What a server buys the vendor is a subscription, not a better result.