Spectral Synthesis

Split sound into a frequency-time map, then rebuild it cell by cell.

Related

HarmonicsTimbreThe software synthesizerContemporary

The question

If a sound is a map of frequency against time, could you edit the map and turn it back into audio? Spectral synthesis treats timbre as a redrawable image.

How it works

A short-time Fourier transform cuts the sound into frames, each yielding frequencies and magnitudes. Stacking the frames produces a spectrogram. Synthesis runs the reverse: magnitudes and phases go through an inverse transform, and frames overlap-add into a waveform. Painting the map edits partials directly; erase a line and a harmonic disappears, stretch one and it rings longer. Phase must stay coherent, or clicks appear between frames. Computation is heavier than subtractive synthesis, but the editing freedom is greater.

Settings and what you hear

Frame length trades time against frequency resolution: long frames give fine frequency detail but smear time; short frames do the opposite. Window shape controls sidelobes, and strong ones let neighbouring partials contaminate each other. Changing magnitude is intuitive; changing phase often adds a metallic or hollow colour. Freeze a spectrum and let it evolve and the result is stretched, with no real attack.

What to listen for

Hear a pad and notice that its highs do not decay but hang in place. Then hear a glockenspiel, where each partial reads as a separate line. Finally hear an effects layer, where broadband noise looks like fog rather than a line.

Common misconceptions

That spectral synthesis is just additive synthesis; that any map edit must be audible, when phase and windowing shape the result; that computing and latency costs can be ignored.

Sources

• Miller Puckette, The Theory and Technique of Electronic Music — spectral analysis checked, not quoted
• Udo Zölzer, DAFX — Digital Audio Effects, spectral processing checked, not quoted

Used for fact-checking only; all prose is original.