Probability Theory 16 | Variance

Variance turns “how far from the mean” into a precise number. After expectation identifies the average location a random variable fluctuates around, variance measures the typical spread by quantifying how much the values deviate from that expectation—using a definition built to behave well mathematically.

The construction starts with the deviation random variable X − E[X]. This new quantity has expectation 0, but it can be positive or negative, so squaring it removes sign and focuses on magnitude. Variance is then defined as Var(X) = E[(X − E[X])²], assuming only that the relevant expectation exists (in particular, that E[X²] is finite). Because variance is itself an expectation, it inherits the same “average of a function” structure as expectation.

Expanding the square yields a more computationally convenient form. Writing (X − E[X])² = X² − 2E[X]·X + (E[X])², and using linearity of expectation plus the fact that E[c] = c for constants, the middle term simplifies and the final result becomes Var(X) = E[X²] − (E[X])². This version is often easier in practice: once E[X] is known, calculating variance reduces to finding E[X²] and subtracting the square of the mean.

For actual calculations, the transcript distinguishes between discrete and continuous settings. In the discrete case, expectations become sums over outcomes weighted by the probability mass function. In the continuous case, expectations become integrals weighted by the probability density function. The only change from the expectation formulas is that the integrand involves X² instead of X.

A first worked example uses a discrete uniform distribution with n equally likely outcomes x₁, …, xₙ, each having probability 1/n. The expectation becomes the arithmetic mean, often written as x̄. Plugging into the variance definition gives Var(X) = (1/n)·∑_{j=1}^n (xⱼ − x̄)², i.e., the average of squared deviations from the mean.

The second example is continuous: an exponential distribution with parameter λ. From earlier results, E[X] = 1/λ. To get variance, the key step is computing E[X²] via the integral ∫₀^∞ x² · λ e^{−λx} dx. Using integration by parts twice, the result is E[X²] = 2/λ². Subtracting (E[X])² = 1/λ² gives Var(X) = 1/λ².

Overall, variance emerges as the standard, well-behaved measure of spread: it is defined through squared deviations, simplifies to E[X²] − (E[X])², and produces concrete formulas for common distributions like the discrete uniform and the exponential.