Probability Theory 14 | Expectation and Change-of-Variables [dark version]

Expectation turns a random variable into a single number: the average value it fluctuates around. For a random variable X, the expectation E[X] is defined as an “abstract integral” over the probability space, written as ∫_Ω X dP (when it exists). In continuous settings, this same idea becomes the familiar mean computed from a probability density function (PDF): E[X] = ∫ x f_X(x) dx, where f_X(x) is nonnegative and integrates to 1. The expectation lands at the “center of mass” of the distribution—like the midpoint of a symmetric density—making it intuitive as the typical value.

The core technical tool behind transforming expectations is the change-of-variables formula for integrals under probability measures. When a new function g is applied to a random variable—forming a new random variable g(X)—the expectation of g(X) can be rewritten using a substitution rule that tracks how probability mass moves. Concretely, the integral over the original sample space can be converted into an integral over the image space (the real line) by using the distribution of X. This is expressed through the pushforward measure: the measure induced on real numbers by X, often denoted P_X. In practice, that means the abstract integral ∫_Ω g(X(ω)) dP(ω) can be rewritten as an integral (or sum) over x ∈ ℝ with respect to P_X.

Once everything is expressed in terms of P_X, the continuous and discrete cases become straightforward. If X has a PDF, the expectation reduces to an ordinary integral with f_X(x). If X is discrete, it becomes a series using the probability mass function (PMF) p_X(x), where p_X(x) = P(X = x). This unified viewpoint—abstract integration plus change of variables—explains why the same expectation formulas appear across different probability models.

The video closes with a concrete example: rolling a fair die once. Here X takes values {1,2,3,4,5,6} with probability 1/6 each, so E[X] is the finite sum 1·(1/6) + 2·(1/6) + … + 6·(1/6). Computing that sum yields 3.5. A key takeaway is that E[X] need not be an actual outcome of the random variable; it can be a value outside the set {1,…,6}. That distinction matters: expectation is a statistical mean of all possible outcomes weighted by their probabilities, not a guaranteed result.

Overall, the message is that expectation is defined abstractly but computed concretely—either by integrals with PDFs or sums with PMFs—because change-of-variables lets probability be transferred from the sample space to the real line via the distribution of X.