DSIP — Visual Interactive Study Guide

1. Digital vs Analog

Analog = continuous: every in-between value exists (a ramp, a slope — like a real sound wave). Digital = discrete: only fixed steps exist (a staircase — you stand on step 1 or 2, never 1.5).

A computer can only store digital values, so every real-world signal (sound, light) must be converted to digital first. Drag the slider to crush the smooth ramp into steps.

2. Frequency & Period

• Frequency = how many full cycles per second, in Hertz (Hz).
• Period = time for one cycle = 1 / frequency.

Why this matters in DSIP: frequency is what Nyquist (Section 3) is measured against — you must sample at least twice the highest frequency present. A faster wave is harder to capture, which is exactly why high-pitched sound and fine image detail need higher sample rates.

Drag the frequency. Notice period is just its reciprocal — they move opposite ways.

3. Sampling & Quantization — the conversion pair

These are the two steps that turn analog into digital. They are different and act on different axes:

• Sampling = how often you measure → acts on the time (x) axis. "10 samples per second" = read the value 10 times each second.
• Quantization = how precisely you record each value → acts on the amplitude (y) axis. Snap each reading to the nearest allowed level.

Drag Sample rate to add/remove dots along time. Drag Levels to make the staircase coarse or fine. Watch what happens when the rate gets too low.

Why exactly 2×? A sine wave has two things to pin down each cycle — its peak and its trough — so you need a little more than two samples per cycle to lock onto its true frequency. Sample any slower and several different waves fit the same dots, so the math can no longer tell them apart. This is the Nyquist–Shannon sampling theorem: capture everything below half the sample rate (the Nyquist frequency), lose everything above it.

What aliasing really is. A frequency that is too high doesn't just disappear — it folds back and reappears as a lower frequency of f_s − f, impersonating a wave that was never there. You see this everywhere: car wheels that spin backwards on video (the wagon-wheel effect), shimmering moiré patterns on striped shirts in photos, and it's why CD audio samples at 44.1 kHz — just over twice the ~20 kHz limit of human hearing. To prevent it, real systems run an anti-aliasing (low-pass) filter that removes the too-high frequencies before sampling.

Go deeper: Nyquist–Shannon theorem (Wikipedia) · The Scientist & Engineer's Guide to DSP, ch.3

4. Bit Depth → Levels (Accuracy vs Memory)

This is the second half of quantization (Section 3): the number of levels you snap each sample to is set by the bit depth. Each extra bit doubles the number of shades you can store. More levels = smoother, more accurate — but more storage. This is the core trade-off: Accuracy ↔ Memory. Too few bits and smooth gradients break into visible stripes (banding) — drag the slider down to 1–2 bits to see it.

5. Images: Pixels, Bit Depth & RGB

An image = a grid of pixels. A 4×4 grid = 16 pixels, stored as a 2D matrix.

• Grayscale: each pixel = one value. 3-bit → 2³ = 8 shades, values 0 (black) … 7 (white).
• Color (RGB): each pixel = three values (Red, Green, Blue), each 0–255. e.g. rgb(128,240,192) = medium red, high green, light blue.

Storage cost follows directly: colour needs 3× the memory of grayscale because every pixel stores three numbers instead of one. That single fact drives the size calculations in Section 9.

Drag the three sliders to mix a color. Click any cell in the grid to paint it.

6. How Cameras Capture Color (Mosaicing)

A pixel "needs" three numbers (R, G, B), but a camera sensor cell can only measure one colour of light. So each cell gets a tiny coloured filter and records just one of the three values; the missing two are guessed from the neighbours by an algorithm. That guessing step is called demosaicing.

The most common layout is the Bayer filter: a repeating 2×2 tile of R G / G B. Notice there are two greens for every red and blue — because the human eye is most sensitive to green (then red, then blue), so the sensor spends its extra detail where your eye cares most.

So a "12-megapixel" photo is really 12 million single-colour samples, and roughly two-thirds of every pixel's colour is interpolated — not directly measured.

7. Image Downsampling (Resizing Down)

Downsampling shrinks an image by keeping fewer pixels — exactly the 2-D version of lowering the sample rate from Section 3. A downsample factor of 2 turns every 2×2 block of pixels into a single pixel, so a 512×512 image becomes 256×256 and stores only a quarter of the pixels.

There are two ways to choose that one output pixel, and the difference is the whole point:

• Subsampling (decimation) — just keep one pixel from each block and throw the rest away. Fast, but it ignores the pixels it drops, so fine detail can fold into fake patterns.
• Averaging (area / box filter) — average each block into its output pixel. This is a low-pass pre-filter (Section 18) that removes detail too fine to survive before dropping it.

Drag factor up and toggle the method. Watch how plain subsampling makes the fine stripes break into shimmering moiré / aliasing, while averaging keeps them smooth — the same Nyquist lesson as audio, now in 2-D.

Go deeper: Downsampling (Wikipedia) · Moiré pattern (Wikipedia)

8. Audio Data-Rate Calculator

Storage per second = samples/sec × bytes/sample. Slide the controls to see the cost.

9. Image Size / Bit-Budget Calculator

An RGB image needs width × height × 3 values. Given a fixed byte budget, how many bits per value can you afford?

10. Discrete Signals & Operations

Continuous uses round brackets x → f(x) = y. Discrete uses square brackets n → f[n], where n = the sample index (often time) and f[n] = the amplitude at that sample. No values exist between whole n.

Pick an operation and compare the original (grey) to the result (blue).

11. Unit Sample / Delta Signal δ[n]

A single spike at 0 — the simplest possible signal, and the building block of every signal.

Any signal can be written as a sum of shifted, scaled deltas. Add some below and watch the composite signal appear.

This "every signal is a stack of shifted spikes" idea is the building-block (sifting) property, and it is what makes the delta so important. Picture a signal as a row of values; each value is simply one impulse sitting at that index, scaled by that value. So any input is just a weighted sum of deltas.

Now the payoff: if a system is linear and time-invariant, feeding it one impulse gives a fixed output called the impulse response h[n]. Because the input is a sum of shifted impulses, the output is the same sum of shifted impulse responses. Measure how the system answers a single spike and you can predict its answer to everything — that summation is convolution (Section 14).

Go deeper: MIT OCW 6.003 — Signals & Systems · Impulse / Dirac delta (Wikipedia)

12. Step Signal U[n]

A signal that is "off," then turns on and stays on.

Two key relationships connect the step to the delta:

These two lines are the discrete version of integration and differentiation. A running sum of impulses keeps adding 1 from n=0 onward, building the flat plateau of the step (like integrating). The first difference — this sample minus the previous one — is zero everywhere the step is flat and jumps to 1 only at the single instant it switches on, handing the impulse back (like differentiating). Summing and differencing are inverse operations, which is exactly why δ and U are inverses of each other.

Go deeper: Unit step function (Wikipedia)

13. Feedback Loop & Geometric Series

A feedback system feeds its output back into its input, multiplied by a relay factor R. The input keeps echoing back, each time smaller — like sound echoing in a room.

Drag R. The bars are successive echoes; the running total converges to x/(1−R).

Where the formula comes from. The output is the input plus a delayed copy of itself: Y = x + R·Y. Collect the Y terms — Y − R·Y = x, so Y(1 − R) = x — and divide to get Y = x/(1 − R). Expanding 1/(1−R) as the geometric series shows the same answer as a stack of fading echoes: the original, then R as loud, then R², and so on forever.

This is a real IIR (infinite impulse response) filter — the maths behind digital echo and reverb. A single input click comes back an infinite number of times, each quieter than the last, because the output is fed back in. It only stays stable while each echo is smaller than the one before.

Go deeper: Geometric series (Wikipedia) · IIR filters (Wikipedia)

14. Convolution (the core concept)

Convolution = slide one signal over another and measure how much they overlap at each position. It's how every filter — blur, sharpen, edge-detect, and every conv-net layer — actually works.

The intuition: at each position, the kernel asks “how much of my shape is present in the signal right here?” Line the two up, multiply point-by-point, add it all into one number, then shift by one and repeat. Where the signal locally matches the kernel the sum is large; where it doesn't, the products cancel toward zero. That single multiply-and-add, swept across every position, is the whole operation.

Press Step to slide the flipped kernel one position at a time. Each output value is the sum of the aligned products.

Edges: near the start and end the kernel hangs off the signal, so you must decide what's "outside" — pad with zeros, repeat the edge value, or wrap around. Different choices give slightly different border pixels.

Fast trick — the diagonal table

Convolving two short lists = multiplying polynomials. Build a multiply table, then sum the diagonals.

Go deeper: 3Blue1Brown — “But what is a convolution?” · Convolution (Wikipedia)

15. Image Convolution Kernels (blur / sharpen / edge)

A 2-D kernel is a tiny weight grid swept over every pixel. At each spot, multiply the 3×3 neighborhood by the weights and add them up — that sum is the new center pixel. Pick a kernel and hover any output cell to see its math.

Read the weights to predict the effect. When the weights sum to 1, a flat region keeps its brightness — the kernel is just a weighted average (blur). When they sum to 0, flat regions cancel to black and only places where neighbours differ — edges — survive. Sharpening is "original + a dose of the edges," which is why its centre weight is bumped to 5 instead of 4.

Two practical notes: at the image border the 3×3 window hangs off the edge, so you pad (repeat or zero the missing pixels) — here we repeat the edge. And the Gaussian blur is separable: a 2-D blur equals a 1-D blur across rows followed by a 1-D blur down columns, which is far cheaper to compute than the full grid.

Go deeper: Explained Visually — Image Kernels · Stanford CS231n — Convolutional Networks

16. 2-D Convolution, Step by Step (9×9 ⊛ 3×3)

The same multiply-add from Section 15, walked through on a full 9×9 image. The 3×3 kernel sits over one neighbourhood, multiplies each pixel by its weight, and the total becomes one output pixel. Then it slides one step right and repeats — across, then down.

Press Step ▶ to walk the window cell by cell (or hover any output cell). The blue box on the left is the current 3×3 window; the matching output cell lights up on the right.

The default edge-detect kernel sums to 0, so flat regions of the image come out black and only the borders of the shape light up — exactly the edge it was built to find. Border pixels reuse the nearest edge value (padding) so the window always has 9 inputs.

17. Spatial Filters: Mean / Median / Max

A spatial filter replaces the centre pixel using its 3×3 neighbourhood — but not every filter is a weighted sum. These order-statistic filters pick a value from the sorted window instead:

• Mean — the average. Smooths noise, but also blurs edges (it's the blur kernel of Section 15).
• Median — the middle value after sorting. Removes salt-and-pepper noise beautifully while keeping edges sharp, because a lone black/white spike is thrown out, not averaged in.
• Max / Min — the brightest / darkest neighbour. Max grows bright regions (dilation); Min grows dark ones (erosion).

The grid below has salt-and-pepper noise (random black/white specks). Switch filters and watch which one cleans it without smearing.

18. Frequency View: Low-pass vs High-pass

Every filter can be read in terms of frequency. Low frequency = slow, smooth change (flat walls, gentle gradients); high frequency = rapid change over a tiny distance (edges, fine texture, noise).

This is why the kernels of Section 15 behave the way they do: a blur is a low-pass filter (it lets the smooth parts through and removes the fast wiggles), while edge-detect and sharpen are high-pass (they throw away the flat background and keep only the rapid changes). Sharpening = original + a dose of the high-pass result.

19. Neuron = Weighted Sum

A single neuron does exactly what a convolution tap (Section 14) does: multiply each input by a learned weight, add them up, add a bias. Stack these and you get a neural network; slide a tiny one over an image and you get a convolutional network. This is the link between classic DSIP and modern AI image processing: the hand-designed kernels of Section 15 become weights the network learns for itself.

20. Intensity Transform S = T(r)

These are point operations: each pixel's new value depends only on its own old intensity r, run through a transfer function T — neighbours are ignored (unlike the kernels of Section 15). The shape of the curve is the operation. Pick a preset and read the shape.

The two everyday ones: Brightness just adds a constant to every pixel — S = r + k. The whole curve shifts up (brighter) or down (darker); values that run past 0 or 255 simply clip. Contrast instead tilts the line around mid-grey (128): S = (r − 128)·g + 128. A steeper slope g pushes darks darker and brights brighter so they separate more; a shallow slope flattens everything toward grey. Watch the strip beside the curve to see the effect on a real grey ramp, not just the line.

Beyond these, two non-linear curves are exam favourites. A log transform S = c·log(1 + r) stretches the dark end and compresses the bright end, pulling faint detail out of shadows. A power-law (gamma) transform S = c·r^γ does the same job with a tunable strength — and is exactly the gamma curve of Section 23. Stretching a narrow band of inputs across the full output range is called contrast stretching.

Go deeper: Gonzalez & Woods — Digital Image Processing (ch.3)

21. Normalization / Requantization

Stretch any range of raw values into a clean 0 … (2^bit − 1) range — used to fit data into a fixed bit depth.

This is min–max contrast stretching: find the darkest and brightest values actually present, map the darkest to 0 and the brightest to the maximum, and spread everything in between. A washed-out image whose values only span 90–160 gets re-stretched across the full 0–255, restoring punch. It's also how you re-fit data into a chosen bit depth (Section 4) — the 2^bit − 1 term sets the new ceiling. The "min" and "max" usually come from reading the image's histogram.

Below, an image whose pixels are cramped into a narrow band. Drag the f_min / f_max window onto that band (and the output bit depth) and watch the input histogram stretch to fill the whole range.

Go deeper: Image normalization (Wikipedia)

22. Histogram Equalization

Min–max stretching (Section 21) only cares about the darkest and brightest values. Histogram equalization is smarter: it looks at how many pixels sit at each value and spreads the crowded values apart, so a dull image where everything clusters in a narrow band gets its tones spread evenly across the full 0 … 255 range.

The trick is the cumulative histogram (CDF) — a running total of pixel counts. Use it as the transfer curve: S = round( CDF(r) × (L−1) ). Where many pixels pile up, the CDF rises steeply, so those values get pushed far apart (more contrast); where few pixels live, the CDF is flat and they barely move.

Result: the output histogram looks flatter and wider than the input, and detail hidden in crowded brightness regions becomes visible. (It can also exaggerate noise — that's the trade-off.)

Go deeper: Histogram equalization (Wikipedia) · Gonzalez & Woods — Digital Image Processing (ch.3)

23. Gamma Correction

Displays and eyes are non-linear, so intensities are reshaped by a power curve before display. Drag γ and watch both the curve and the gradient strip change.

The easy way to picture it. Think of the 256 brightness codes as a limited budget. A straight (linear) line spends them evenly — but your eye barely notices the difference between two bright codes, so half of them are wasted. Gamma bends the line so more codes land in the dark region the eye actually scrutinises. Encoding raises darks up the scale (giving them room), and the display later pushes them back down — so the picture looks right while the shadows keep their detail.

The real reason gamma exists. Human vision is far more sensitive to changes in dark tones than bright ones — doubling a dim light is very noticeable; doubling an already-bright one barely registers. If you stored brightness with evenly-spaced steps (linear), you'd waste most of your codes on highlights the eye can't tell apart, while the shadows — where the eye is fussy — would show ugly banding.

So images are saved with an encoding gamma of about 1/2.2 (≈ 0.45), which packs more code values into the dark region where they're needed; the display then applies the inverse decoding gamma of about 2.2 to recover true light output. The two cancel, and the limited 8 bits are spent where your eye actually cares. The common sRGB standard follows roughly a 2.2 curve (with a small linear segment near black). That's why the exponent is 1/γ, not γ — you're pre-distorting to undo the display.

Go deeper: Cambridge in Colour — Understanding Gamma · Gamma correction (Wikipedia)

24. The Imaging Pipeline

Every concept above lives somewhere on this path from scene to perception. Use it as a one-glance revision map: if you can place each topic on the pipeline and say why it happens there, you understand the course.