Statistics vary from sample to sample
A parameter describes a whole population, such as the population mean μ or the population proportion p. A statistic is calculated from a sample, such as the sample mean x̄ or the sample proportion p̂.
Take a different random sample and you will usually get a different value of the statistic. This is sampling variability. It is not a mistake: it is what random sampling does. To use a statistic well, you need to know how much it tends to vary, and what values are typical. That is what a sampling distribution tells you.
The sampling distribution of a statistic is the distribution of the values of that statistic for all possible samples of a given size from a given population.
Three different distributions
Students often mix these up. Keep them apart by asking, “What is one dot on the graph?”
| Distribution | One value is… | Example (call durations) |
|---|---|---|
| Population distribution | one individual in the population | the length of one call, out of all 2,000 calls |
| Distribution of one sample | one individual in your sample | the length of one of the 40 calls you sampled |
| Sampling distribution | the statistic from one whole sample | the mean length x̄ of one sample of 40 calls |
The first two show how individuals vary. The sampling distribution shows how a statistic varies from sample to sample.
Worked example 1: an exact sampling distribution
Question. A school keeps exactly five beehives (fictional). Last summer their honey yields were 2, 3, 5, 8 and 12 kg, so the population mean is μ = 30 ÷ 5 = 6 kg. (a) List every possible random sample of 2 hives (without replacement) and find the sampling distribution of x̄. (b) Describe it, and compare it with samples of 3 hives.
(a) There are 10 possible samples of 2 hives. Each is equally likely in a simple random sample.
| Sample (kg) | 2, 3 | 2, 5 | 2, 8 | 2, 12 | 3, 5 | 3, 8 | 3, 12 | 5, 8 | 5, 12 | 8, 12 |
|---|---|---|---|---|---|---|---|---|---|---|
| x̄ (kg) | 2.5 | 3.5 | 5.0 | 7.0 | 4.0 | 5.5 | 7.5 | 6.5 | 8.5 | 10.0 |
So the sampling distribution of x̄ for n = 2 has these 10 values, each with probability 1/10 = 0.1.
(b)
- Centre. The mean of the 10 sample means is 60 ÷ 10 = 6.0 kg, exactly μ.
- Variability. The sample means run from 2.5 kg to 10.0 kg (range 7.5 kg). Their standard deviation, treating the 10 values as the whole distribution, is about 2.22 kg. The population’s standard deviation is about 3.63 kg, so the sample means vary less than individual hives.
- A probability. P(5 ≤ x̄ ≤ 7) = 4/10 = 0.4. So a sample of 2 hives gives a mean within 1 kg of μ only 40% of the time.
- Samples of 3 hives. There are again 10 possible samples. Their means run from 3.33 kg to 8.33 kg (range 5.0 kg), with mean 6.0 kg and standard deviation about 1.48 kg. Now P(5 ≤ x̄ ≤ 7) = 5/10 = 0.5.
Interpretation. Both sampling distributions are centred on the population mean of 6 kg. With 3 hives per sample, the sample means are less spread out, so a single sample mean is more likely to be close to μ.
Check. The 10 sample means for n = 2 add to 60, and 60 ÷ 10 = 6 = μ. (Topic 3.1 explains why a statistic whose sampling distribution is centred on the parameter is called unbiased.)
Simulating a sampling distribution
Real populations are too large to list every sample. Instead, simulate:
- Assume a value for the parameter, for example p = 0.75 or a model for the population.
- Generate a random sample of size n from that population with technology.
- Calculate and record the statistic for that sample.
- Repeat steps 2 and 3 many times (hundreds or thousands).
- Graph the recorded values. This simulated distribution approximates the sampling distribution.
Describe the result as you would any distribution: shape, centre and variability, in context. Then you can ask whether an observed statistic is typical or unusual if the assumed parameter value is true.
Worked example 2: is the germination claim believable?
Question. A fictional seed company claims that 75% of its tomato seeds germinate. A gardener plants a random sample of 40 seeds and 24 germinate, so p̂ = 24 ÷ 40 = 0.60. (a) Describe how to simulate the sampling distribution of p̂, assuming the claim is true. (b) Describe the simulated distribution in Figure 1. (c) Is p̂ = 0.60 unusual if the claim is true?
(a) One trial: use technology to generate 40 random integers from 1 to 100. Let 1–75 represent a seed that germinates and 76–100 one that does not. Count the germinating seeds and divide by 40 to get p̂. Record it. Repeat for 200 trials.
(b) The simulated distribution of p̂ is roughly symmetric and unimodal. It is centred near 0.75 (the mean of the 200 values is 0.7624 and the median is 0.75). The values vary from 0.575 to 0.90, with a standard deviation of about 0.063. So, if 75% of the seeds germinate, samples of 40 seeds usually give a sample proportion within about 0.10 of 0.75.
(c) Only 3 of the 200 simulated samples (1.5%) gave p̂ of 0.60 or less. So a result as low as the gardener’s would be unusual if the claim were true. This gives the gardener reason to doubt the company’s claim of 75%. (Unit 3 turns this reasoning into a formal significance test.)
Check. The simulated distribution is centred near the assumed p = 0.75, as it should be. Another set of 200 simulations would give slightly different counts, but a similar picture.
Randomization distributions for experiments
In an experiment, the randomness comes from random assignment of subjects to treatments, not from random sampling. A randomization distribution shows how much a statistic, such as a difference in means, varies just because of which subjects happened to land in which group.
- Write each subject’s response value on a card.
- Shuffle the cards and deal them at random into groups of the original sizes. This is one reallocation.
- Calculate and record the statistic (for example, mean of group A − mean of group B).
- Repeat many times and graph the results.
This keeps every response exactly as observed and only changes the group labels. So it shows the values the statistic would take if the treatment made no difference to anyone’s response. If the observed statistic sits far out in the tail, chance assignment alone is an unlikely explanation.
Worked example 3: a memory-app experiment
Question. Ten fictional volunteers were randomly assigned, 5 to practise with a memory app for a week and 5 to a control group. Each then took a recall test (scores out of 20). App: 14, 17, 12, 18, 16. Control: 11, 13, 10, 15, 12. (a) Find the observed difference in means. (b) Describe one reallocation. (c) Use the 1,000 reallocations in Figure 2 to judge the result.
(a) App mean = 77 ÷ 5 = 15.4. Control mean = 61 ÷ 5 = 12.2. Observed difference (app − control) = 3.2 points.
(b) Put the 10 scores on cards. Shuffle and deal 5 cards to “app” and 5 to “control”. Calculate (mean of “app” cards) − (mean of “control” cards) and record it.
(c) The randomization distribution is roughly symmetric, centred near 0 (mean −0.08), and the differences vary from −4.4 to 4.4 points, with a standard deviation of about 1.70 points. Only 37 of the 1,000 reallocations (3.7%) gave a difference of 3.2 points or more. If the app made no difference, a gap this large would happen by chance assignment only rarely. So the result suggests the app may improve recall for people like these volunteers. (With only 10 volunteers, this is a small study.)
Check. The distribution is centred near 0. That is what we expect when the group labels are shuffled at random and the responses do not change.
The central limit theorem
The central limit theorem (CLT) says: the sampling distribution of the mean of a random sample can be approximated by a normal distribution, and the larger the sample, the better the approximation. This holds even when the population itself is not normal.
Figure 3 shows the CLT at work. A fictional helpline has a population of 2,000 call durations. The population is strongly skewed to the right: most calls are short, a few are very long. Its mean is 3.83 minutes, its median 2.60 minutes and its standard deviation 3.87 minutes. For each sample size, technology took 1,000 random samples and recorded each sample mean.
Worked example 4: describing the CLT simulation
Question. Use Figure 3 to describe how the sampling distribution of x̄ changes as n increases from 2 to 40.
| n | Mean of the 1,000 x̄ values | Standard deviation of the x̄ values | Shape |
|---|---|---|---|
| population | 3.83 (μ) | 3.87 (σ) | strongly right-skewed |
| 2 | 3.81 | 2.66 | right-skewed |
| 10 | 3.81 | 1.26 | slightly right-skewed |
| 40 | 3.82 | 0.60 | roughly symmetric, bell-shaped |
- Shape. As n increases, the skew fades. For n = 40 the histogram is close to a normal curve. This is the CLT.
- Centre. For every n, the sample means are centred close to the population mean of 3.83 minutes.
- Variability. The sample means vary less as n grows: their standard deviation falls from 2.66 minutes (n = 2) to 0.60 minutes (n = 40). Larger samples give sample means that are closer to μ more of the time.
Interpretation. For random samples of 40 calls, the mean call duration is approximately normally distributed around 3.83 minutes, and most sample means lie between about 2.6 and 5.0 minutes, even though individual calls are very skewed and run from under 1 minute to over 30.
Check. The individual calls inside one sample of 40 are still right-skewed, like the population. Only the distribution of the sample means becomes approximately normal. Topic 4.1 gives a formula for the standard deviation of x̄ and a rule of thumb for how large n must be.
Common misconceptions
- “The sampling distribution is the distribution of my sample.” Your sample’s data show individuals. The sampling distribution shows the statistic over all possible samples.
- “The CLT makes the population, or my sample, normal.” It does not. Only the distribution of sample means becomes approximately normal.
- “More simulated samples means a bigger sample size.” The number of repetitions (200, 1,000) improves the picture of the sampling distribution. The sample size n is the number of individuals in each sample, and only n changes the spread of the statistic.
- “A larger sample size changes the centre.” The centre stays near the parameter; it is the variability that shrinks.
- “An unusual result proves the claim is false.” It gives evidence against the claim. Unusual results do occasionally happen when a claim is true.
- Reallocating with different group sizes. A randomization must keep the original group sizes and the observed responses; only the labels move.
- Describing a simulated distribution with no context. Name the statistic, the sample size and the assumed parameter value, and describe shape, centre and variability.
Where this leads
Sampling distributions are the foundation of inference. In Topic 3.1 you will use them to decide whether a statistic is a good estimator of a parameter, and Units 3 and 4 add formulas for the centre and spread of the sampling distributions of p̂ and x̄. Try the practice questions now, then use the revision notes and the checklist to consolidate. For the normal model itself, look back at the Topic 2.11 study guide.
Quick questions
Does the central limit theorem make my data normal?
No. It is about the distribution of the sample mean over many samples. The values inside one sample still look like the population, so a sample from a skewed population is still skewed, however large it is.
How many simulated samples do I need?
Enough for the shape to settle down: a few hundred or more is common. More simulated samples give a better picture of the sampling distribution. They do not change the sample size n, which is the number of individuals in each sample.