Session 40 - Probability Distribution Functions - PDF, PMF & CDF | DSMP 2023

Probability distributions become the bridge between raw outcomes and usable probability—especially when data analysts need to estimate what values are likely, how they’re spread, and how to compute probabilities for ranges. The session starts by laying the groundwork: random variables replace algebra’s “unknown value” with a variable that can take multiple possible outcomes from a random experiment. A coin toss example is used to show how a random variable is defined over a sample space, and the session distinguishes discrete random variables (finite countable outcomes like die results) from continuous random variables (outcomes over a range, like exam marks).

From there, the core idea of a probability distribution is introduced as a structured listing of all possible outcomes paired with their probabilities. For discrete cases, this is demonstrated through a table-style probability distribution for sums from dice rolls, highlighting that probabilities differ by outcome (some sums occur more often than others). The session then points out a practical limitation: manually building tables becomes unwieldy as the number of outcomes grows, and it becomes impossible for continuous variables where outcomes are effectively infinite. The solution is to use a mathematical relationship—a probability distribution function—so probabilities can be computed and plotted without enumerating every outcome.

The session emphasizes that probability distribution functions (PDF/PMF/CDF) are not just theoretical graphs; they provide two major benefits. First, the shape of the distribution reveals where the data “concentrates” (e.g., most students’ marks cluster around certain values). Second, if the observed distribution matches a known “famous” distribution (like Normal, Uniform, Bernoulli, Binomial), analysts can reuse established results rather than starting from scratch. The session also notes that each distribution comes with parameters (numerical tuning knobs) that control location and scale, changing the graph’s shape.

A significant portion then distinguishes discrete and continuous distribution functions. For discrete random variables, the probability mass function (PMF) gives the probability at each specific outcome, with the session stressing two constraints: probabilities must be nonnegative and sum to 1. It also demonstrates how PMF can be approximated empirically by running simulations (e.g., rolling dice many times) and comparing observed frequencies to theoretical probabilities.

For continuous random variables, the session introduces the probability density function (PDF) and explains why it cannot be interpreted like PMF. In continuous settings, the probability of any exact single value is effectively zero, so the meaningful quantity is probability over an interval. The session uses the “area under the curve” idea: integrating the PDF over a range yields the probability that the random variable falls within that interval. It also introduces the cumulative distribution function (CDF) as the running probability up to a value (probability of being less than or equal to x), and connects CDF and PDF through calculus intuition: differentiating CDF gives PDF, while integrating PDF gives CDF.

Finally, the session moves toward practical estimation. When the true distribution is unknown, density estimation techniques are needed to approximate the PDF. Two approaches are contrasted: parametric density estimation (assume a distribution family like Normal and estimate parameters such as mean and standard deviation) and non-parametric density estimation (make fewer assumptions and estimate the density directly from data). The session highlights kernel density estimation (KDE) as a common non-parametric method, where each data point contributes a “bump” (often Gaussian) and the bumps are summed to form a smooth estimated density. The session closes by previewing that future coverage will focus on applying these ideas to real datasets, including practical plotting of distributions and extending from 1D to 2D density plots.