Justifying a Claim Based on a Confidence Interval for Two Proportions: Study Guide (Statistics 3.11)

What the interval is estimating

In Topic 3.10 you built intervals of the form

(p̂₁ − p̂₂) ± z √[ p̂₁(1 − p̂₁)/n₁ + p̂₂(1 − p̂₂)/n₂ ]*

This topic is about what such an interval means and how to use it to answer a question.

The parameter is the difference p₁ − p₂. It is a fixed number: the proportion in population 1 minus the proportion in population 2. In an experiment, the two “populations” are the two treatments: the proportion of subjects like these who would respond with treatment 1, minus the proportion with treatment 2.

The interval is built from two random samples (or one random assignment). Change the samples and p̂₁ − p̂₂ changes, so the interval moves. Any one interval you calculate may or may not capture p₁ − p₂, and you cannot tell which, because the true difference is unknown. That is why you describe a single interval with confidence, and keep the word probability for the method.

Interpreting the interval in context

A full interpretation of a C% interval for p₁ − p₂ has four parts:

1. the confidence level, C%;
2. the interval, from the lower limit a to the upper limit b;
3. the order of subtraction, so the reader knows what a positive value means;
4. the parameter in context: the response and both populations (or both treatments).

A reliable template:

We are C% confident that the interval from a to b captures the difference (population 1 − population 2) in the proportions of all [individuals] who [response].

The signs of the endpoints carry meaning:

• A positive value means population 1 has the larger proportion.
• A negative value means population 2 has the larger proportion.
• 0 means no difference.

It often helps to translate into percentage points. An interval of (0.058, 0.182) says population 1’s proportion is between about 5.8 and 18.2 percentage points higher than population 2’s. Say “percentage points”, not “percent”: a rise from 36% to 48% is 12 percentage points, not 12%.

Two parts are easy to forget. The interpretation is about the populations, not the samples: the interval always contains p̂₁ − p̂₂, because that is its centre. And it is about one number, the difference in proportions. It says nothing about individual people.

Interpreting the confidence level

The confidence level describes the method:

If we took many pairs of random samples, with the same sample sizes, from the same two populations, and built a C% interval from each pair, about C% of those intervals would capture the true difference p₁ − p₂.

In an experiment, replace “pairs of random samples” with “random assignments of the same subjects to the two treatments”.

So 95% confidence does not mean “a 95% chance that p₁ − p₂ is in this interval”. Once the interval is calculated it either captures the difference or it does not. The 95% is the long-run success rate of the procedure that produced it.

Using the interval to justify a claim

The interval gives a range of plausible values for p₁ − p₂. The key value is 0, because p₁ − p₂ = 0 means the two proportions are equal.

What the interval looks like	What you can conclude
Contains 0 (one endpoint negative, one positive)	0 is plausible. There is not convincing evidence of a difference between the two population proportions.
Every value positive (0 is below the interval)	0 is not plausible. There is convincing evidence of a difference, and in particular that p₁ > p₂.
Every value negative (0 is above the interval)	0 is not plausible. There is convincing evidence of a difference, and in particular that p₁ < p₂.

The same idea works for claims about the size of the difference. If someone claims p₁ is 10 percentage points higher than p₂, check whether 0.10 is inside the interval. Inside means plausible (no convincing evidence against the claim, but not proof). Outside means there is convincing evidence against it. For a claim such as “at least 20 percentage points higher”, look at where the whole interval lies relative to 0.20.

Finish with a sentence in context that refers to the interval and its confidence level.

Worked example 1: independent random samples, three claims

Context (fictional). An energy regulator compares two districts, Oldmere (about 30,000 households) and Kestwick (about 26,000 households). Independent random samples found that 240 of 500 Oldmere households and 162 of 450 Kestwick households have a smart meter. The conditions are met: independent random samples; 500 ≤ 3,000 and 450 ≤ 2,600 (10% of each population); counts 240, 260, 162 and 288 are all at least 10.

(a) Construct and interpret a 95% interval for p_O − p_K.

1. p̂_O = 240 ÷ 500 = 0.48 and p̂_K = 162 ÷ 450 = 0.36. Point estimate 0.12.
2. SE = √(0.48 × 0.52 ÷ 500 + 0.36 × 0.64 ÷ 450) ≈ 0.0318.
3. MOE = 1.960 × 0.0318 ≈ 0.0623.
4. Interval: 0.12 ± 0.0623 = (0.058, 0.182).

Interpretation. We are 95% confident that the interval from 0.058 to 0.182 captures the difference (Oldmere − Kestwick) in the proportions of all households with a smart meter. In words: Oldmere’s proportion is plausibly between about 5.8 and 18.2 percentage points higher.

(b) Interpret the confidence level. If the regulator took many pairs of independent random samples of 500 Oldmere households and 450 Kestwick households and built a 95% interval from each pair, about 95% of the intervals would capture the true difference in the proportions with a smart meter.

(c) Judge three claims.

1. “The proportions differ.” The interval does not contain 0. There is convincing evidence of a difference.
2. “A higher proportion of Oldmere households have a smart meter.” Every value in the interval is positive. There is convincing evidence that p_O > p_K.
3. “Oldmere’s proportion is at least 20 percentage points higher.” 0.20 is above the whole interval. The interval gives convincing evidence against this claim. (A claim of “10 percentage points higher” would be plausible, because 0.10 is inside.)

Check. Reversing the order gives (−0.182, −0.058), which is all negative: Kestwick’s proportion is lower. Same conclusion, different words. The 90% interval (0.068, 0.172) and the 99% interval (0.038, 0.202) also exclude 0. Note that the 99% interval includes 0.20, so claim 3 is judged differently at 99% confidence; state the confidence level with every conclusion.

Worked example 2: a randomized experiment where the interval contains 0

Context (fictional). A council wants more households to use their food-waste caddies. It randomly assigns 400 households to receive the caddy with either a printed instruction booklet (200 households) or a picture-only leaflet (200 households). After a month, 118 booklet households and 106 leaflet households use the caddy every week. The conditions are met: random assignment; the 10% condition is not needed for an experiment; counts 118, 82, 106 and 94 are all at least 10.

(a) p̂_B = 0.59 and p̂_L = 0.53, a difference of 0.06. SE = √(0.59 × 0.41 ÷ 200 + 0.53 × 0.47 ÷ 200) ≈ 0.04955. MOE = 1.960 × 0.04955 ≈ 0.0971. The 95% interval for p_B − p_L is (−0.037, 0.157).

Interpretation. We are 95% confident that the interval from −0.037 to 0.157 captures the difference (booklet − leaflet) in the proportions of households like these that would use their caddy every week.

(b) Does the booklet lead to more weekly use? The interval contains 0, so 0 is a plausible value for p_B − p_L. The experiment does not give convincing evidence that the booklet changes the proportion of households using their caddy weekly.

(c) A councillor says: “So the booklet makes no difference.” This goes too far. The interval also contains values as large as 0.157: the booklet might raise weekly use by about 15 percentage points, or lower it by about 4. The data cannot tell these apart. “No convincing evidence of a difference” is not the same as “evidence of no difference”.

(d) What would help? A larger experiment gives a narrower interval. If the sample proportions stayed at 0.59 and 0.53, about 600 households per treatment would give a margin of error of about 0.056, small enough to exclude 0. And because the treatments were randomly assigned, an interval that excluded 0 would support a cause-and-effect conclusion: the booklet causes a change in use for households like these.

Worked example 3: judging interpretations, and how the confidence level matters

Context (fictional). A bank selects independent random samples of 700 customers from each of its two largest branches. At the Riverside branch 315 rate the service “excellent” (0.45); at the Market Street branch 273 do (0.39). The conditions are met. The 95% interval for p_R − p_M is (0.008, 0.112) and the 99% interval is (−0.008, 0.128).

Which of these sentences are correct?

1. “We are 95% confident that the interval from 0.008 to 0.112 captures the difference (Riverside − Market Street) in the proportions of all customers at each branch who rate the service excellent.” Correct. Level, endpoints, order and both populations are all there.
2. “There is a 95% probability that the true difference is between 0.008 and 0.112.” Wrong. The true difference is fixed. The 95% describes the method.
3. “95% of Riverside customers are between 0.8 and 11.2 percentage points more satisfied than Market Street customers.” Wrong. The interval estimates one difference in proportions, not a range for individual customers.
4. “Riverside’s sample proportion is between 0.008 and 0.112 higher.” Wrong. The sample difference is exactly 0.06. The interval is about the populations.

Is there convincing evidence that the branches differ?

• At 95% confidence: the interval (0.008, 0.112) does not contain 0, so yes, there is convincing evidence that a higher proportion of all Riverside customers rate the service excellent.
• At 99% confidence: the interval (−0.008, 0.128) contains 0, so no, there is not convincing evidence of a difference.

The data have not changed. A higher confidence level gives a wider interval, which here just reaches past 0. Decide the confidence level before looking at the data, and state it with the conclusion.

Scope. These are independent random samples, not an experiment. So the conclusion can be generalised to all customers of the two branches, but it cannot say why the branches differ.

Common misconceptions

• “There is a 95% chance p₁ − p₂ is in this interval.” After calculation, the interval either captures the difference or not. The 95% belongs to the method.
• “The interval contains 0, so the proportions are equal.” 0 is plausible, not proved. Other values in the interval are plausible too.
• Giving no order of subtraction. Without it, a reader cannot tell which group a positive value favours.
• “The interval is (0.058, 0.182), so the difference is 5.8% to 18.2%.” These are percentage points, not percentages of something.
• Interpreting the interval for the samples. You already know p̂₁ − p̂₂. The interval is about the populations or treatments.
• Judging the claim from the point estimate. “The difference is 0.06, which is not 0, so there is a difference” ignores sampling variability. Use the whole interval.
• Claiming cause from independent samples. Only random assignment of treatments supports a cause-and-effect conclusion.
• Changing the confidence level after seeing the result to get the conclusion you want.

Where this leads

Next, Topic 3.12 sets up a significance test that asks the same question, “is there a difference?”, in a different way: see Setting Up a Test for the Difference Between Two Population Proportions. Topic 3.13 then carries out that test. Try the practice questions now, then use the revision notes and the checklist. To review how the interval is built, return to Topic 3.10; for the one-proportion version of these ideas, see Topic 3.4.