Markov's Inequality

Markov's inequality is the simplest probability bound: if X >= 0 and E[X] exists, then P(X >= a) <= E[X] / a for any a > 0. The only requirements are that X is non-negative and has a finite mean. No knowledge of the distribution shape is needed.

The proof is elegant. Since X >= 0, E[X] = integral from 0 to infinity of x * f(x) dx >= integral from a to infinity of x * f(x) dx >= a * integral from a to infinity of f(x) dx = a * P(X >= a). Dividing both sides by a gives the result.

Example: If expected claim cost is $500, then P(claim >= $2000) <= 500/2000 = 0.25. The actual probability is likely much smaller (for example, 0.082 for an exponential distribution), but Markov provides a guaranteed upper bound that holds for any non-negative distribution with that mean.

Markov's inequality can also be applied to functions of random variables. If g(X) >= 0, then P(g(X) >= a) <= E[g(X)] / a. Taking g(x) = (x - mu)^2 and a = k^2 * sigma^2 leads directly to Chebyshev's inequality, showing that Chebyshev is a consequence of Markov.

Chebyshev's Inequality

For any random variable X with mean mu and finite variance sigma^2, Chebyshev's inequality states P(|X - mu| >= k * sigma) <= 1/k^2 for any k > 0. Equivalently, P(|X - mu| >= a) <= sigma^2 / a^2 for any a > 0.

Chebyshev is tighter than Markov because it uses both the mean and the variance. It tells you that the probability of being more than k standard deviations from the mean is at most 1/k^2, regardless of the shape of the distribution. For k = 2, P(|X - mu| >= 2*sigma) <= 0.25. For k = 3, P(|X - mu| >= 3*sigma) <= 1/9 = 0.111.

Example: If claims have mean 500 and standard deviation 100, then P(|X - 500| >= 300) <= (100/300)^2 = 1/9. This means at least 8/9 (about 88.9%) of claims fall between 200 and 800. Compare this to the normal distribution, where P(|X - 500| >= 300) = P(|Z| >= 3) = 0.0027. The Chebyshev bound is conservative but distribution-free.

The one-sided version of Chebyshev's inequality (sometimes called the Cantelli inequality) states P(X - mu >= k*sigma) <= 1/(1 + k^2). This is tighter than dividing the two-sided Chebyshev bound by 2. While less commonly tested on Exam P, it occasionally appears in problems asking for one-sided bounds.

Applications of Chebyshev to Sample Means

Chebyshev's inequality applied to the sample mean X_bar gives P(|X_bar - mu| >= epsilon) <= sigma^2/(n * epsilon^2). This bound decreases with n, proving the Weak Law of Large Numbers: X_bar converges to mu in probability.

On Exam P, this application answers questions like: "How many observations are needed to ensure the sample mean is within epsilon of the true mean with probability at least 1 - delta?" Setting sigma^2/(n * epsilon^2) <= delta gives n >= sigma^2/(delta * epsilon^2).

Example: An actuary wants to estimate the mean claim amount to within $50 with 95% confidence. If sigma = 200, then n >= 200^2 / (0.05 * 50^2) = 40000 / 125 = 320 claims. Using the normal approximation (which is tighter), the required n would be (1.96 * 200/50)^2 = 61.5, so about 62. The Chebyshev bound is conservative but always valid, while the normal approximation requires the CLT to hold.

Jensen's Inequality

Jensen's inequality relates the expectation of a function to the function of the expectation. If g is a convex function, then E[g(X)] >= g(E[X]). If g is concave, the inequality reverses: E[g(X)] <= g(E[X]).

Recall that a function g is convex if g''(x) >= 0 for all x (the function curves upward), and concave if g''(x) <= 0 (the function curves downward). Common convex functions include x^2, e^x, and |x|. Common concave functions include ln(x), sqrt(x), and x^p for 0 < p < 1.

Important applications on Exam P:

Since x^2 is convex: E[X^2] >= (E[X])^2, confirming Var(X) = E[X^2] - (E[X])^2 >= 0
Since ln(x) is concave: E[ln(X)] <= ln(E[X]), with equality only when X is constant
Since e^x is convex: E[e^X] >= e^(E[X]), meaning the MGF M(1) >= e^mu
Since sqrt(x) is concave: E[sqrt(X)] <= sqrt(E[X])

Jensen's inequality is useful for bounding expectations of transformed random variables when the exact distribution is unknown. It also provides quick sanity checks on calculations.

Comparing the Three Inequalities

Each inequality has different requirements and gives different types of bounds:

Markov requires: X >= 0 and finite E[X]. It bounds P(X >= a) using only the mean.
Chebyshev requires: finite E[X] and Var(X). It bounds P(|X - mu| >= a) using the mean and variance. It is always tighter than Markov for the same type of bound.
Jensen is not a probability bound but an expectation inequality. It requires knowing whether a function is convex or concave and is used to compare E[g(X)] with g(E[X]).

On Exam P, choose the inequality based on what information you have. If you only know the mean and X >= 0, use Markov. If you also know the variance, use Chebyshev for a tighter bound. If you need to bound an expectation of a transformed variable, use Jensen.

Common Pitfalls

A frequent error with Markov's inequality is applying it when X is not non-negative. The inequality P(X >= a) <= E[X]/a requires X >= 0. If X can be negative, apply Markov to |X| or (X - c)^2 instead. For instance, applying Markov to (X - mu)^2 gives Chebyshev.

With Chebyshev, a common mistake is using the wrong form of the inequality. P(|X - mu| >= a) <= sigma^2/a^2 bounds the probability of deviating from the mean by at least a, not by at most a. To bound P(|X - mu| < a), use the complement: P(|X - mu| < a) >= 1 - sigma^2/a^2.

For Jensen's inequality, the most common error is applying the inequality in the wrong direction. Always check whether the function is convex (E[g(X)] >= g(E[X])) or concave (E[g(X)] <= g(E[X])) before writing the inequality. A quick way to check: if the function curves upward (like x^2 or e^x), it is convex, and the expectation of the function exceeds the function of the expectation.

Study Tips

For Exam P, memorize both forms of Chebyshev (using k*sigma and using a), know Markov's inequality and its requirement of non-negativity, and be comfortable applying Jensen to common convex and concave functions. Practice problems that ask you to bound probabilities using Chebyshev, determine sample sizes using the Chebyshev bound on the sample mean, and compare E[g(X)] with g(E[X]) using Jensen. These inequality problems typically take two to three minutes and test conceptual understanding as much as computation. About one to two inequality problems appear on each Exam P administration.

Probability Inequalities: Chebyshev, Markov, and Jensen