Lesson 3.5: Selecting Models and Linear Combinations of Random Variables

Introduction

In this lesson, students will explore the different probability distributions that can be used to model real-world situations. We will focus on the binomial, normal, Poisson, and exponential distributions. Furthermore, we will learn how to evaluate the mean and variance of linear combinations of independent random variables. The objectives are designed to guide students through judging the appropriateness of various models for given scenarios and calculating key statistics for combinations of random variables.

Learning Objectives

• Judging the validity of binomial, normal, Poisson, and exponential models in particular real-world situations.
• Evaluating the mean and variance of linear combinations of independent random variables.
• Evaluating probabilities for linear combinations of two or more independent normal distributions in practical situations.
• Choosing an appropriate distribution for a context and justifying the choice from the modeling assumptions.
• Finding the mean and variance of a linear combination of independent random variables.

Understanding Probability Distributions

Probability distributions are a statistical framework that describes how probabilities are assigned to different outcomes of a random event. Each distribution is governed by certain parameters which affect its shape, mean, and variance. The choice of a probability distribution largely depends on the nature of the data and the process generating the data.

Common Probability Distributions

1. Binomial Distribution: Used to model the number of successes in a fixed number of independent Bernoulli trials (trials that have two outcomes: success or failure). The binomial distribution is parameterized by $n$, the number of trials, and $p$, the probability of success in each trial. The probability mass function is given by:

$$P(X = k) = \binom{n}{k} p^k (1-p)^{n-k}$$

for $k = 0, 1, 2, \ldots, n$.

1. Normal Distribution: A continuous probability distribution that is symmetric about the mean, indicating that data near the mean are more frequent in occurrence than data far from the mean. It is defined by its mean $\mu$ and variance $\sigma^2$. The probability density function is:

$$f(x) = \frac{1}{\sigma \sqrt{2\pi}} e^{-\frac{(x - \mu)^2}{2\sigma^2}}$$

1. Poisson Distribution: Models the number of events occurring in a fixed interval of time or space. The events must occur with a known constant mean rate and independently of the time since the last event. It is characterized by $\lambda$, the average rate at which events occur. The probability mass function is:

$$P(X = k) = \frac{\lambda^k e^{-\lambda}}{k!}$$

1. Exponential Distribution: A continuous probability distribution that describes the time between events in a Poisson process. It has a constant hazard rate and is characterized by the rate parameter $\lambda$. The probability density function is:

$$f(x) = \lambda e^{-\lambda x}, \quad x \geq 0$$

Worked Example: Binomial Distribution

Suppose we want to know the probability of getting exactly 3 heads when flipping a fair coin 5 times. Here, a head is considered a success.

• Given: $n = 5$, $p = 0.5$, and $k = 3$.
• We can calculate the probability as follows:

$$P(X = 3) = \binom{5}{3} (0.5)^3 (0.5)^{5-3} = \binom{5}{3} (0.5)^5$$

$$P(X = 3) = 10 \cdot \frac{1}{32} = \frac{10}{32} = \frac{5}{16}$$

Validating Probability Models

To determine which distribution is appropriate for a certain real-world scenario, we must consider the underlying assumptions of each distribution:

• Binomial models require a fixed number of trials with two outcomes and constant probability.
• Normal models require a large enough sample size, where the Central Limit Theorem applies, for distributions that are not inherently normal.
• Poisson models apply to events in fixed intervals and assume independence.
• Exponential models apply to the time until the next event in a Poisson process.

Common Misconceptions

1. Confusing Binomial and Normal Distributions: The binomial distribution can be approximated by a normal distribution only under certain conditions (typically when $np \geq 5$ and $n(1-p) \geq 5$).
2. Use of Poisson for Large n: While Poisson can approximate binomial (under certain conditions), it is primarily for counts in fixed intervals.
3. Ignoring Parameters: Each distribution's parameters must match the context of the problem; failure to specify these correctly can lead to incorrect conclusions.

Linear Combinations of Random Variables

When dealing with independent random variables, we can form linear combinations to find new random variables. If $X_1, X_2, \ldots, X_n$ are independent random variables with means $\mu_i$ and variances $\sigma_i^2$, the linear combination can be expressed as:

$$Y = a_1 X_1 + a_2 X_2 + \ldots + a_n X_n$$

where $a_i$ are constants.

Mean and Variance of a Linear Combination

The mean and variance of $Y$ can be given as:

• Mean:

$$E[Y] = a_1 E[X_1] + a_2 E[X_2] + \ldots + a_n E[X_n]$$

• Variance:

$$\text{Var}(Y) = a_1^2 \text{Var}(X_1) + a_2^2 \text{Var}(X_2) + \ldots + a_n^2 \text{Var}(X_n)$$

Worked Example: Linear Combination of Random Variables

Suppose we have two independent random variables, $X_1$ and $X_2$, with the following characteristics:

• $X_1$ has $E[X_1] = 20$ and $\text{Var}(X_1) = 4$.
• $X_2$ has $E[X_2] = 30$ and $\text{Var}(X_2) = 9$.

Let’s find the expected value and variance of the linear combination:

$$Y = 2X_1 + 3X_2$$

• Mean:

$$E[Y] = 2E[X_1] + 3E[X_2] = 2(20) + 3(30) = 40 + 90 = 130$$

• Variance:

$$\text{Var}(Y) = 2^2 \text{Var}(X_1) + 3^2 \text{Var}(X_2) = 4(4) + 9(9) = 16 + 81 = 97$$

Conclusion

In this lesson, students has learned about the various probability distributions and how to determine their appropriateness for different real-world contexts. We explored the characteristics and formulas of binomial, normal, Poisson, and exponential distributions. Additionally, students learned how to compute the mean and variance of linear combinations of independent random variables. These concepts are fundamental for applying statistical models to practical problems.

Study Notes

• Probability distributions model the likelihood of different outcomes.
• Binomial distribution applies to a fixed number of trials with two outcomes.
• Normal distribution describes data that is symmetrically distributed about the mean.
• Poisson distribution is used for counting events in fixed intervals.
• Exponential distribution models the time between independent events.
• For linear combinations of independent random variables:
• $ E[Y] = a_1 E[X_1] + a_2 E[X_2} + \ldots $
• $ \text{Var}(Y) = a_1^2 \text{Var}(X_1) + a_2^2 \text{Var}(X_2) + \ldots $
• Always check the assumptions of the chosen probability model against the context of the problem.