MDM Reference  •  Random variables

Random Variables and Their Distributions

A random variable is a rule that assigns a number to each outcome of an experiment. The study of such rules, and of the distribution they induce on the line, is the grammar in which the rest of statistics is written.

From outcomes to numbers

A random variable carries the irregular space of outcomes onto the ordered line, where arithmetic is available. Once a number is attached to an outcome, expectations may be formed, inequalities may be used, and limits may be taken. The burden of this convenience is that the rule must be measurable, a condition that is satisfied by every rule met in ordinary practice.

Distribution and density

The distribution function gives the probability that the variable does not exceed a stated level. It is non-decreasing, tends to zero on the far left and to one on the far right, and is continuous from the right. For a continuous variable it is the integral of a density; for a discrete variable it is a step function whose jumps are the point probabilities. The density is not a probability, though it is often read as if it were, and that reading leads the beginner astray.

Moments and their meaning

• The expectation locates the centre of the distribution.
• The variance measures its spread about that centre.
• The skewness records the lean of the distribution to one side.
• The kurtosis records the weight carried in the tails.

Moments exist only when the integrals that define them converge, and heavy-tailed distributions are precisely those for which the higher moments fail to exist. The reader is advised to test existence before quoting a moment.

Transformations

A change of variable carries a density from one scale to another, the density being divided by the absolute value of the derivative of the rule. This single rule underlies the lognormal, the chi-square and many other families, each of which arises as a transformed member of a simpler one. Mastery of the change of variable rule removes much of the labour of remembering separate formulas.

Independence and conditioning

Two variables are independent when their joint distribution is the product of their margins. Independence is stronger than the absence of correlation, and the two coincide only in special families. Conditioning, the reverse of the product rule, updates a distribution once a second variable is seen, and it is the formal content of learning from observation.

Families met in practice

The binomial, the Poisson, the exponential and the Gaussian recur so often that they deserve separate study. The first two describe counts, the third describes waiting, and the fourth occurs whenever many small independent contributions are added together. The next cell of this reference treats that fourth family in its own right.

A caution on reading a density

The height of a density at a point is not the probability of that point, since a single point carries no probability for a continuous variable. Only the area under the density over an interval carries probability, and the height merely sets how densely that area is packed. The caution is repeated here because the height is so easily read as a probability by a beginner.