Probability Distributions Review for Exam MAS-I
Review essential probability distributions and their properties for Exam MAS-I preparation.
Discrete Distributions Overview
Exam MAS-I requires thorough knowledge of discrete distributions and the relationships connecting them. The Bernoulli distribution models a single trial with success probability p: P(X=1) = p and P(X=0) = 1-p, with E[X] = p and Var(X) = p(1-p). The Binomial distribution extends this to n independent trials: X ~ Bin(n,p) has P(X=k) = C(n,k) * p^k * (1-p)^(n-k), with E[X] = np and Var(X) = np(1-p). The Binomial is the workhorse distribution for modeling fixed-trial count data.
The Poisson distribution models rare event counts with parameter lambda: P(X=k) = e^(-lambda) * lambda^k / k!, with the defining property E[X] = Var(X) = lambda. The equal mean and variance is a diagnostic feature; when observed data shows variance exceeding the mean (overdispersion), the Poisson is inadequate. The Poisson arises as the limit of Binomial(n,p) as n approaches infinity and p approaches 0 with np = lambda held constant, making it ideal for modeling rare events in large populations.
Overdispersed Count Distributions
The Negative Binomial distribution handles overdispersion with parameters r and p. It can be interpreted as the number of failures before r successes, or as a Gamma mixture of Poisson distributions. Its mean is r(1-p)/p and variance is r(1-p)/p^2, so the variance always exceeds the mean. The Geometric distribution is the special case with r=1, modeling the number of failures before the first success.
The Hypergeometric distribution models sampling without replacement from a finite population. If a population contains N items of which K are "successes," and n items are drawn without replacement, the number of successes follows Hypergeometric(N, K, n). Its mean is nK/N and its variance is nK(N-K)(N-n) / (N^2(N-1)). The factor (N-n)/(N-1) is the finite population correction, which makes the variance smaller than the corresponding Binomial variance.
Continuous Distributions: Normal Family
The Normal distribution N(mu, sigma^2) is the foundation of statistical inference. Its PDF is f(x) = (1/(sigma*sqrt(2*pi))) * exp(-(x-mu)^2 / (2*sigma^2)). Key properties include: the sum of independent normals is normal, any linear combination of normal variables is normal, and the Central Limit Theorem guarantees that sample means from any distribution with finite variance converge to normal as the sample size grows.
The Chi-squared distribution with k degrees of freedom is the sum of k squared standard normal variables. Its mean is k and variance is 2k. It appears in variance estimation and goodness-of-fit testing. The t-distribution with k degrees of freedom arises as Z/sqrt(V/k) where Z is standard normal and V is chi-squared with k degrees of freedom. It is used for inference on means when the population variance is unknown. The F-distribution with (d1, d2) degrees of freedom is the ratio of two independent chi-squared variables divided by their respective degrees of freedom, and it appears in ANOVA and regression testing.
Continuous Distributions: Modeling Applications
The Exponential distribution with mean theta has PDF f(x) = (1/theta) * exp(-x/theta) and CDF F(x) = 1 - exp(-x/theta). Its defining property is memorylessness: P(X > s+t | X > s) = P(X > t). The hazard rate is constant at 1/theta. The Gamma distribution generalizes the exponential: with shape alpha and rate 1/theta, it represents the sum of alpha independent exponential random variables. Its mean is alpha*theta and variance is alpha*theta^2.
The Lognormal distribution arises when ln(X) follows a normal distribution. If ln(X) ~ N(mu, sigma^2), then E[X] = exp(mu + sigma^2/2) and Var(X) = exp(2*mu + sigma^2) * (exp(sigma^2) - 1). Lognormal distributions model multiplicative processes and are commonly used for modeling insurance losses and financial returns. The Beta distribution on [0,1] with parameters alpha and beta serves as a conjugate prior for the Binomial proportion and is useful for modeling probabilities or proportions.
Moment Generating Functions
The moment generating function (MGF) M_X(t) = E[e^(tX)] uniquely determines a distribution and provides a convenient way to derive moments: E[X^k] = M_X^(k)(0), the kth derivative evaluated at t=0. Key MGFs include: Poisson has M(t) = exp(lambda(e^t - 1)), Normal has M(t) = exp(mu*t + sigma^2*t^2/2), Exponential has M(t) = 1/(1 - theta*t) for t < 1/theta, and Gamma has M(t) = (1 - theta*t)^(-alpha).
MGFs are particularly useful for proving that sums of independent random variables follow specific distributions. For example, the sum of independent Poisson variables with parameters lambda_1 and lambda_2 has MGF exp((lambda_1 + lambda_2)(e^t - 1)), which is Poisson with parameter lambda_1 + lambda_2. Similarly, the sum of independent normals is normal, and the sum of independent gammas (with the same rate parameter) is gamma.
Relationships Between Distributions
Understanding the connections between distributions is essential for Exam MAS-I. The Binomial converges to Poisson as n grows with np = lambda. The Negative Binomial is a Gamma mixture of Poissons. The Chi-squared with k degrees of freedom is Gamma(k/2, 2). The Exponential is Gamma(1, theta). The t-distribution converges to Normal as degrees of freedom increase. The F-distribution with (1, k) degrees of freedom equals the square of a t-distribution with k degrees of freedom.
These relationships appear frequently in exam problems, either explicitly (derive one distribution from another) or implicitly (recognize that a given scenario follows a specific distribution based on its construction).
Common Pitfalls
A frequent mistake is confusing parameterizations. The Exponential distribution can be parameterized by the mean theta or by the rate lambda = 1/theta, and different textbooks use different conventions. Similarly, the Gamma distribution can use a rate or a scale parameter. Always check which parameterization the problem uses before applying formulas.
Another common error is misapplying the CLT. The CLT applies to sample means (or sums) of independent, identically distributed random variables. It does not apply to individual observations, to dependent data, or to distributions without finite variance. Also, the CLT provides an approximation that improves with sample size; for small samples, the approximation may be poor.
Study Tips
For Exam MAS-I, create a comprehensive reference sheet listing each distribution's PMF or PDF, mean, variance, MGF, and key properties. Organize it by family (count distributions, continuous distributions, sampling distributions) and note the relationships between them. Practice deriving moments from MGFs and recognizing distributions from their construction. The distributions reviewed here appear throughout the MAS-I syllabus in regression, ANOVA, Bayesian analysis, and time series contexts, so a solid foundation pays dividends across all topics.