Sampling Distributions for the Difference Between Sample Proportions: Study Guide (Statistics 3.9)

Why compare two proportions?

Many statistical questions compare two groups. Do more adults in one town recycle food waste than in another? Does a text reminder get more patients to their appointments than an email? Each question is about a difference between two population proportions, p1 − p2.

You cannot usually measure everyone, so you take a sample from each group and calculate two sample proportions, p̂1 and p̂2. Their difference, p̂1 − p̂2, is your estimate of p1 − p2. Just like one sample proportion (Topic 3.2), this difference changes from sample to sample. The distribution of p̂1 − p̂2 over all possible pairs of samples is its sampling distribution. This topic gives you its centre, its spread and the conditions under which its shape is approximately normal.

Keep the notation straight:

Symbol	Meaning	Type
p1, p2	proportion of successes in population 1 and population 2	parameters
N1, N2	sizes of population 1 and population 2	population sizes
n1, n2	sizes of sample 1 and sample 2	sample sizes
p̂1, p̂2	proportion of successes in sample 1 and sample 2	statistics
p̂1 − p̂2	difference in sample proportions	statistic (estimates p1 − p2)

Choose an order of subtraction at the start, say what it is, and keep it. “Brackenford − Thornley” is clear; “the difference” on its own is not.

Mean and standard deviation of p̂1 − p̂2

Suppose the two samples are independent of each other and the observations within each sample are independent. Then the sampling distribution of p̂1 − p̂2 has:

• Mean: μ(p̂1 − p̂2) = p1 − p2
• Standard deviation: σ(p̂1 − p̂2) = √[ p1(1 − p1)/n1 + p2(1 − p2)/n2 ]

Why these make sense:

1. The mean. From Topic 3.2, the mean of p̂1 is p1 and the mean of p̂2 is p2. On average, the difference is the difference of the averages: p1 − p2. So p̂1 − p̂2 is an unbiased estimator of p1 − p2.
2. The standard deviation. The quantity under the square root is the variance of p̂1, p1(1 − p1)/n1, plus the variance of p̂2, p2(1 − p2)/n2. The variances add, even though the proportions are subtracted. Each sample brings its own random error, and those errors do not cancel. (Background: for two independent random variables X and Y, the variance of X − Y is the variance of X plus the variance of Y.)

Two warnings follow from the formula:

• Do not add or subtract the two standard deviations. Add the two variances, then take one square root.
• The standard deviation of the difference is always larger than the standard deviation of either sample proportion alone.

Larger samples still give less variability. If you multiply both sample sizes by 4, both variances are divided by 4, so the standard deviation is halved.

The conditions and what each one is for

The course groups the checks into three conditions. Verify each one in context, with numbers.

Condition	What to check	What it guarantees
Randomization	Two independent random samples, one from each population; or, in an experiment, treatments randomly assigned to the experimental units.	The samples are independent of each other and p̂1 − p̂2 is unbiased.
10%	When sampling without replacement: n1 ≤ 10% of N1 and n2 ≤ 10% of N2.	Observations within each sample are close enough to independent for the standard deviation formula to be accurate.
Large counts	All four expected counts are at least 10: n1p1, n1(1 − p1), n2p2 and n2(1 − p2).	The sampling distribution of p̂1 − p̂2 is approximately normal.

Points that cost marks:

• Experiments. If the data come from an experiment, the independence requirement is met by random assignment of the treatments. The 10% condition is not needed, because no one is sampled from a larger population. You still need large counts if you want a normal model.
• Four counts, not two. Check expected successes and failures in both samples.
• Expected counts use p. In this topic the population proportions are given (or assumed), so the counts use p1 and p2. In Topic 3.10, where they are unknown, you will use the observed counts instead.
• “Independent” means two things. The two samples must not be linked (for example, the same people measured twice), and the observations within each sample must be independent (random selection plus the 10% condition).

Picturing the sampling distribution

Worked example 1: mean, standard deviation and conditions

Question. Two fictional towns run food-waste collections. In Brackenford (36,000 adults), 58% of adults use the collection every week. In Thornley (25,000 adults), 45% do. A researcher will take independent simple random samples of 150 Brackenford adults and 180 Thornley adults. Let p̂B and p̂T be the sample proportions who use the collection every week.

(a) Find the mean and standard deviation of the sampling distribution of p̂B − p̂T. (b) Check the conditions. (c) Interpret the mean and the standard deviation in context.

(a)

1. Mean: pB − pT = 0.58 − 0.45 = 0.13.
2. Variance of p̂B: 0.58 × 0.42 ÷ 150 = 0.001624.
3. Variance of p̂T: 0.45 × 0.55 ÷ 180 = 0.001375.
4. Add the variances: 0.001624 + 0.001375 = 0.002999.
5. Standard deviation: √0.002999 ≈ 0.0548.

(b)

1. Randomization: the two samples are independent simple random samples, one from each town. ✓
2. 10%: 150 ≤ 10% of 36,000 = 3,600 and 180 ≤ 10% of 25,000 = 2,500. ✓ So the standard deviation formula is accurate.
3. Large counts: 150 × 0.58 = 87, 150 × 0.42 = 63, 180 × 0.45 = 81 and 180 × 0.55 = 99. All four are at least 10. ✓ So the sampling distribution of p̂B − p̂T is approximately normal.

(c)

• Mean: if the researcher took many pairs of random samples (150 from Brackenford, 180 from Thornley), the differences in sample proportions (Brackenford − Thornley) who use the collection weekly would average 0.13, the true difference between the towns.
• Standard deviation: in such pairs of samples, the difference in sample proportions (Brackenford − Thornley) typically varies from the true difference of 0.13 by about 0.0548, or about 5.5 percentage points.

Check. The two separate standard deviations are about 0.0403 and 0.0371. Our answer, 0.0548, is larger than each but smaller than their sum (0.0774), as it must be when variances add.

Worked example 2: probabilities with the normal model

Question. Use the sampling distribution from Worked example 1. (a) Find the probability that the sample proportion in Brackenford is at least 0.20 greater than the sample proportion in Thornley. (b) Find the probability that the Thornley sample proportion is greater than the Brackenford one. (c) The researcher’s samples contain 96 weekly users out of 150 in Brackenford and 72 out of 180 in Thornley. Is this difference surprising if the stated town proportions are correct?

(a) The large-counts condition is met, so use a normal model with mean 0.13 and standard deviation 0.05476.

1. z = (0.20 − 0.13) ÷ 0.05476 ≈ 1.2783.
2. P(p̂B − p̂T ≥ 0.20) = P(z ≥ 1.2783) ≈ 0.1006.

Interpretation: if the stated proportions are true, about 10% of pairs of random samples of these sizes would show a Brackenford proportion at least 20 percentage points above the Thornley proportion.

(b) “Thornley greater” means p̂B − p̂T < 0.

1. z = (0 − 0.13) ÷ 0.05476 ≈ −2.3740.
2. P(p̂B − p̂T < 0) ≈ 0.0088.

So in fewer than 1 in 100 pairs of samples would the order of the towns be reversed. This is the small area to the left of the “no difference” line in Figure 1.

(c) p̂B = 96 ÷ 150 = 0.64 and p̂T = 72 ÷ 180 = 0.40, so p̂B − p̂T = 0.24.

1. z = (0.24 − 0.13) ÷ 0.05476 ≈ 2.0088.
2. P(p̂B − p̂T ≥ 0.24) ≈ 0.0223.

If the true difference were 0.13, a sample difference of 0.24 or more would happen in only about 2% of pairs of samples. That is fairly unusual, so the samples give some reason to doubt that the true difference is as small as 0.13. They do not prove it: unusual samples do happen. Formal tests for a difference come in Topics 3.12 and 3.13.

Check. normalcdf(0.20, 1, 0.13, 0.05476) ≈ 0.1006 and normalcdf(−1, 0, 0.13, 0.05476) ≈ 0.0088.

Worked example 3: an experiment

Question. A fictional health clinic runs an experiment with 240 patients who have booked an appointment. It randomly assigns 120 to receive a text reminder and 120 to receive an email reminder. Suppose that, for patients like these, 85% attend after a text reminder and 75% attend after an email reminder. Let p̂T and p̂E be the proportions of each group who attend.

(a) Check the conditions for a normal model for p̂T − p̂E. (b) Find the mean and standard deviation. (c) Find the probability that the email group’s attendance rate is at least as high as the text group’s. (d) What would this probability become with 240 patients in each group?

(a)

1. Randomization: the treatments (text or email) were randomly assigned to the patients. ✓ The 10% condition is not needed, because this is an experiment, not a sample from a population.
2. Large counts: 120 × 0.85 = 102, 120 × 0.15 = 18, 120 × 0.75 = 90 and 120 × 0.25 = 30. All at least 10. ✓

(b) Mean = 0.85 − 0.75 = 0.10. Standard deviation = √(0.85 × 0.15 ÷ 120 + 0.75 × 0.25 ÷ 120) ≈ 0.0512.

(c) “Email at least as high” means p̂T − p̂E ≤ 0. z = (0 − 0.10) ÷ 0.0512 ≈ −1.9518, so the probability is about 0.0255. Even if text reminders really work better by 10 percentage points, about 2.5% of experiments like this would show email doing as well or better.

(d) With 240 per group, both variances halve, so the standard deviation is divided by √2: it becomes about 0.0362. Then z ≈ −2.7603 and the probability falls to about 0.0029. Larger groups make a misleading result much less likely.

When the large-counts condition fails

Suppose a fictional factory has two production lines. Line A makes 10% faulty parts and Line B makes 4%, and you take random samples of 150 parts from each line. For Line B, n2p2 = 150 × 0.04 = 6, which is less than 10. The sampling distribution of p̂2 is skewed to the right, so the distribution of p̂1 − p̂2 is not reliably normal. The mean and standard deviation formulas still hold (they need only independence), but do not use a normal model for probabilities. A larger sample from Line B fixes this: n2 = 250 gives n2p2 = 10.

Common misconceptions

• “Subtract the standard deviations” or “subtract the variances”. Add the variances: √[p1(1 − p1)/n1 + p2(1 − p2)/n2]. Subtracting can even give zero spread when the two groups match, which is impossible.
• Using n1 + n2 as one sample size. Each variance uses its own sample size.
• Checking only two counts. You need four: successes and failures in each sample.
• Checking the 10% condition for an experiment. Random assignment replaces it.
• “Independent samples” when the same people are in both groups. Measuring one group twice gives paired data, and this formula does not apply.
• Not stating the order of subtraction. A mean of −0.13 and a mean of 0.13 describe the same situation in different orders. Say which group comes first.
• Interpreting without both populations. “The difference varies by about 0.055” is incomplete. Name both towns, the variable and the sample sizes.
• “A small probability proves the stated proportions are wrong.” It gives reason to doubt them; it is not proof.

Where this leads

In Topic 3.10 the population proportions are unknown. You will use p̂1 and p̂2 in the same standard deviation formula to build a confidence interval for p1 − p2: see Constructing a Confidence Interval for a Difference in Two Proportions. First try the practice questions, then use the revision notes and the checklist. If one sample proportion still feels shaky, revisit Topic 3.2. For the errors that tests can make, look back at Topic 3.8.