Live Spectrogram and Spectrum Analyzer

What a spectrogram shows

A spectrogram is a picture of sound. Time runs from left to right, frequency runs up, and how loud each frequency is at each moment is a colour. It is built from short-time Fourier transforms: the sound is cut into short overlapping slices, each slice is tapered to zero at both ends by a window, and a fast Fourier transform turns it into one column of levels. At the default FFT size of 4096 samples at 48 kHz, each column covers 85.3 ms of sound, its levels are 11.7 Hz apart, and a new column starts every 21.3 ms, so about 47 arrive every second.

Above the picture is the newest 40 ms of the waveform itself, held still on a rising zero crossing so that a steady tone does not drift, as the trigger of an oscilloscope holds it. To the right is the live spectrum: the newest column drawn sideways on the same frequency axis, so each peak lines up with the band it is drawing, or on a phone laid flat underneath. On a wider screen a keyboard from C1 to C8 runs beside the log axis and lights the key nearest the pitch.

The colours are Magma, the colour map Nathaniel J. Smith and Stéfan van der Walt made for matplotlib and released under CC0. It runs from black at the floor through violet and red to pale yellow at the ceiling, getting brighter evenly all the way, so the picture still reads in greyscale and for the common kinds of colour blindness. With the default range of −100 to 0 dBFS, violet is about −70 dBFS and orange about −25. Everything blue is yours, the crosshair and the peak marker, and everything amber is pitch: the pitch trace and the note at its newest end, the harmonic ladder and the lit key.

Under the picture, every demo has a card showing its own spectrogram in miniature, analysed from the same samples, so you can see what each one draws before you choose it. The demos run silently, and the picture is the same either way. Turn on Play sound to hear one: everything this page plays, demos and files alike, is scaled so its loudest moment peaks at −18 dBFS and fades in and out over 20 ms, so nothing starts with a bang. The phone keypad sounds one key at a time, so a key you press takes over from the one already sounding instead of adding to it. Here is the showreel, second by second, at the default settings.

• 0 to 3.45 s, three sung vowels. Heed on C3 has a fundamental of 130.8 Hz, wavering by 2 percent either way five and a half times a second, the vibrato, with a harmonic at every whole multiple above it. Its brightest line is the second harmonic, 261.6 Hz, almost on heed’s first formant at 270 Hz, and a second bright band, the 17th and 18th harmonics at 2224 and 2355 Hz, marks its second formant at 2290 Hz. Hod on E3 lights up a block from 659 to 989 Hz, its fourth to sixth harmonics, where its first two formants, 730 and 1090 Hz, sit close together. Who’d on G3 is brightest at its fundamental, 196 Hz, and fades fast above 1 kHz.
• 3.85 to 5.45 s, a whistle. One thin line rising from 1.2 to 2.8 kHz, because a whistle is very nearly a pure tone.
• 5.9 to 7.1 s, the phone keys 1, 2 and 3. Each is two lines at once. The lower stays at 697 Hz, the keypad’s top row, while the upper steps from 1209 to 1336 to 1477 Hz, one column to the next.
• 7.6 to 11.2 s, a chirp from 50 Hz to 16 kHz. It climbs 8.32 octaves at a steady 2.31 octaves a second, so on the log axis it is a straight line. Its low end is thicker, because the bins are 11.7 Hz apart: near 50 Hz a line a few bins wide spans half an octave of the log axis, while above 1 kHz the line is about a tenth of an octave thick, most of that the chirp’s climb during one window.

Using your microphone

Press Use your microphone and your browser asks whether this page may use it, unless you have allowed it before. The page asks for the microphone with echo cancellation, noise suppression and automatic gain control switched off, because each of them changes what is heard: noise suppression removes steady tones, a whistle among them, and automatic gain control evens out loud and quiet. Some phones keep one of them on anyway, and when the browser reports that, the stage says which.

The sound is analysed here, in your browser, as it arrives. This page’s code keeps the picture and the last fraction of a second it is working on, and never records, stores or sends the sound anywhere. The page address holds your settings and the demo you chose, and never anything that was heard.

While the page is listening, a pill reads Listening on this device. Stop, Pause and Space, switching to the demos or to a file, hiding the tab and leaving the page all stop every track of the stream, which is what puts out your browser’s own microphone indicator. From the microphone a column is taken every animation frame and stamped with the audio clock, so if the page stalls, the gap shows as a gap instead of squeezing time together.

An audio file works the same way, without any permission. Choose one, or drop it on the screen: anything your browser can decode, up to 10 minutes and 200 MB. It is decoded and analysed here and is not uploaded, and the whole file is drawn as the analysis goes. Then press Play to hear it, and tap or drag on the picture to move the playhead.

Pitch, harmonics and the missing fundamental

A voice, a string or a pipe makes harmonics: lines at whole multiples of one frequency, the fundamental, which is the pitch you hear. A note on E3, 164.8 Hz, has lines at 329.6, 494.4 and 659.3 Hz and on up. The loudest of them is often not the fundamental, and on hod it is usually the fourth. So the Pitch readout does not read the loudest line. It uses the YIN method of de Cheveigné and Kawahara (2002), which works in time rather than frequency: it slides about 43 ms of signal along itself and finds the shortest shift at which it matches, which is the period, and the pitch is one over that. It searches from 50 Hz to 2 kHz and names a pitch only when the match passes a threshold of 0.12, a little looser than the 0.1 of the paper’s own tests; otherwise the readout says unclear. The amber trace across the picture is the same reading, column by column.

Because it measures repetition, YIN hears what the ear hears when the fundamental itself is missing. Harmonics 2 to 6 of 200 Hz, from 400 to 1200 Hz with nothing at 200 Hz, still repeat every 5 ms, and read 200 Hz. That is the missing fundamental, and it is why a voice on an old telephone line, which carried little below 300 Hz, still sounded at its true pitch. Turn on Harmonics and put the crosshair on a line: rungs appear at 2 to 10 times its frequency, and when a stack of lines sits on the rungs, it is one sound. The square, sawtooth and triangle demos are built from the same coefficients as the Fourier series visualiser, so their ladders of odd and even harmonics are the ones it adds up, and a string or a pipe sounds its harmonics because of the standing waves the standing wave simulator draws.

Repetition can fool it too. A phone key is two tones chosen not to be harmonics of one note, so that speech is unlikely to imitate them, yet two steady tones nearly repeat at some shift, and the Pitch readout names a pitch for every key: the showreel’s keys 1, 2 and 3 read about 239, 674 and 731 Hz, though they play no tone at any of them. Key A comes closest to repeating: 697 and 1633 Hz are almost exactly the third and seventh harmonics of 233 Hz, and the Pitch readout names 233.1 Hz.

Vowels and formants

Every vowel sung on the same note has the same harmonics; what changes is which of them are loud. The air from your vocal folds to your lips is a tube, and it resonates at a few frequencies, the formants, which boost the harmonics near them. Moving the tongue and the lips changes the tube’s shape and moves the formants. Peterson and Barney measured them in 1952 for 76 speakers, and these are their averages for the 33 men:

The demo builds each vowel the same way: a source at the vocal folds whose harmonics fall by 12 dB an octave, the radiation from the lips, which lifts them by 6 dB an octave, and three resonators from Klatt’s 1980 speech synthesiser, set at those averages with bandwidths of 80, 100 and 150 Hz chosen for the demo. While a vowel sounds, the drawing in the corner of the screen shows where the tongue is. High and forward for heed leaves a wide throat and a small space in the mouth, so F1 is low and F2 high. Low and back for hod narrows the throat and opens the mouth, which raises F1 and brings F2 down towards it. High and back with the lips rounded for who’d lengthens the tube and lowers both. The larynx hangs from the hyoid, the small bone you can find in the head and neck explorer.

Time against frequency resolution

A column’s bins are Δf = fs/N apart, and its window lasts N/fs. Both come from the one number, the FFT size N, so finer frequency costs coarser time and the other way round. At 48 kHz:

Two tones show as two lines only when they are about two bins apart, because a Hann window spreads each tone across about four bins. At 48 kHz, 440 and 446 Hz show as two steady lines only at 16384, and only just. At every smaller size they merge into one line whose brightness pulses six times a second, which is what beats look like, and the pulsing is deeper the smaller the size. The Beats demo, 440 and 444 Hz, is closer still: it pulses four times a second at every size, and even at 16384, where its tones are 1.4 bins apart, it never settles into two steady lines.

A large size also blurs anything quick. The edges of the phone keys and the wobble of the vibrato smear at 16384, where one window lasts 341 ms. The chirp climbs 0.79 of an octave inside one such window, so it thickens along its whole length, though its low end, with bins now 2.93 Hz apart, comes out sharper than at 4096. Change the FFT size while a demo runs: new columns take the new size at once, and a dashed line labelled with the new bin spacing marks where it changed. Set the window to rectangular and every tone grows a wide skirt either side. That is leakage, which a tapered window like Hann is there to reduce.

Hum, aliasing and other things you will see

Mains hum is a line at 50 Hz in the UK, Europe, India and Australia, or 60 Hz in the US and Canada, often with harmonics at 100 and 150 Hz or at 120 and 180 Hz, picked up by a cable or a charger near the microphone. On the keyboard 50 Hz is G1 +35 ¢, and 60 Hz is B1 −49 ¢, almost exactly between two notes.

Aliasing. Samples cannot hold anything above half their rate, the Nyquist frequency, so a tone above it reappears below it, folded back as if by a mirror: sampled at fs, a tone at f shows at |f − k fs|, where k is the whole number nearest f/fs. The Aliasing sweep demo samples a rising tone at the rate you set, 8 kHz to begin with, with no filter in front. The line climbs to the dashed Nyquist line at 4 kHz and folds back down while the tone itself, the faint dashed line, carries on up, so a 5 kHz tone shows at 3 kHz. Real recorders filter out everything above the Nyquist frequency before they sample, for exactly this reason, and the sampling and aliasing visualiser takes the fold apart slowly.

Noise near the top. High up a log axis one row of the picture spans many bins, and it shows the loudest of them, which is what keeps a single thin line from fading into its neighbours. For a hiss that flatters the top rows: white noise, the same power in every hertz, looks brighter the higher it goes, and pink noise, whose power in each hertz falls 3 dB an octave, looks flatter than it is.

Other marks. A bright vertical stripe across every frequency is a click: something that changes in an instant contains every frequency at once. A band along the very bottom is rumble, from wind or from handling the microphone; on the linear and mel axes, which start at 0 Hz, a line right on the bottom edge can also be an offset in the signal. A line that bends smoothly down as something passes is the Doppler effect, which the Doppler effect simulator works out.

What this analyser leaves out

• Your microphone and speakers. Levels are in dBFS, decibels relative to the largest value the samples can hold, not sound pressure in the room: the same voice reads louder close to the microphone, or through a more sensitive one. Built-in microphones are usually less sensitive at the lowest and highest frequencies, so the picture shows what reached the samples rather than what was in the air. The decibel calculator converts between decibel scales.
• The device’s sample rate. The microphone is analysed at whatever rate the device runs, often 48 or 44.1 kHz. A file is decoded at that rate once the page has played a sound or opened the microphone, and at 48 kHz before then, resampled by the browser if it was made at another. The Nyquist frequency is half of it, and the top of the axis stops there.
• Processing a phone may keep on. The page asks for echo cancellation, noise suppression and automatic gain control to be off. Some phones keep one on regardless; the stage says so when the browser admits it, and not every browser does.
• More than one pitch. YIN finds one period. Two voices, a chord or a strong echo give it several, and it reports the louder voice, a longer period they share, or none: C, E and G sung together on middle C read C2, two octaves below the lowest of them.
• The room. Echoes smear every line to the right, and a noisy room raises the floor.

Common mistakes

• Reading the loudest line as the pitch. On hod the loudest line is usually the fourth harmonic, 659 Hz, two octaves above the 164.8 Hz you hear. Read the Pitch readout or follow the amber trace.
• Raising the FFT size to see more. A larger size separates close frequencies but smears quick changes. Speech changes every few tens of milliseconds and reads best from 1024 to 4096; a held note can take 8192 or 16384.
• Taking dBFS for loudness. 0 dBFS is the largest value the samples can hold, not a sound level, and −20 dBFS is a tenth of that amplitude, whether it is a whisper close up or a shout across a room.
• Taking a harmonic for a second note. A stack of evenly spaced lines is one sound. Turn on Harmonics and check that the lines sit on the rungs.
• Believing an aliased line. A line that bounces off a fixed height as it rises is folding at the Nyquist frequency, not a real tone coming down.