Carrying Out a Test for the Difference Between Two Proportions: Study Guide (Statistics 3.13)

From set-up to decision

In Topic 3.12 you set up a test for the difference between two population proportions: you defined p₁ and p₂, wrote hypotheses and checked conditions. This topic finishes the job. You calculate the test statistic, find and interpret the p-value, make a decision and answer the investigative question.

The four parts are the same as for one proportion in Topic 3.7. Keep them in this order in every written answer:

1. Hypotheses. Define p₁ and p₂ in context (the proportion of what, in which population or treatment group). State H₀: p₁ = p₂ (or p₁ − p₂ = 0) and Hₐ with <, > or ≠. State α.
2. Method and conditions. Name the procedure: a two-sample z-test for the difference between two population proportions. Check randomization, the 10% condition (for samples drawn without replacement) and the normality condition using the pooled proportion.
3. Calculations. Find p̂₁, p̂₂, the pooled proportion p̂c, the test statistic z and the p-value.
4. Conclusion. Compare the p-value with α, state the decision and say what it means in context.

This guide concentrates on parts 3 and 4. The worked examples show all four parts so you can see a full answer.

The pooled proportion

The null hypothesis says the two population proportions are equal. A test always asks: if H₀ were true, how surprising would these data be? So every calculation is done in a world where p₁ = p₂.

If the two proportions really are equal, both samples are estimating the same number. The best single estimate of that number uses all the data together. This is the combined, or pooled, sample proportion:

p̂c = (x₁ + x₂) / (n₁ + n₂) = (n₁p̂₁ + n₂p̂₂) / (n₁ + n₂)

Here x₁ and x₂ are the numbers of successes and n₁ and n₂ are the sample sizes. In words: add the successes, add the sample sizes, and divide.

Notice that p̂c is not the simple average of p̂₁ and p̂₂ unless n₁ = n₂. The larger sample has more weight, because it contains more individuals.

You met p̂c in Topic 3.12, where it was used to check the normality condition: n₁p̂c, n₁(1 − p̂c), n₂p̂c and n₂(1 − p̂c) must all be at least 10. Now it also appears in the test statistic.

The test statistic

The test statistic measures how far the observed difference in sample proportions is from the difference stated in H₀ (zero), in standard-error units:

z = (p̂₁ − p̂₂ − 0) / √[p̂c(1 − p̂c)(1/n₁ + 1/n₂)]

• The numerator is the observed difference minus the null difference. The “− 0” is a reminder that H₀ says the difference is zero.
• The denominator is the standard deviation of the sampling distribution of p̂₁ − p̂₂ if H₀ is true, estimated with p̂c.
• The sign matters. A positive z means p̂₁ is above p̂₂; a negative z means p̂₁ is below p̂₂.

If H₀ is true and the conditions are met, z has approximately a standard normal distribution. This is the null distribution of the test statistic.

Finding the p-value

The p-value is the probability, assuming H₀ is true, of a test statistic at least as extreme as the one observed, in the direction of Hₐ. Find it from the standard normal distribution with a table or technology:

Alternative	p-value	Area under the standard normal curve
Hₐ: p₁ > p₂ (p₁ − p₂ > 0)	P(Z ≥ z)	right tail beyond z
Hₐ: p₁ < p₂ (p₁ − p₂ < 0)	P(Z ≤ z)	left tail beyond z
Hₐ: p₁ ≠ p₂ (p₁ − p₂ ≠ 0)	2 × P(Z ≥ |z|)	both tails beyond ±|z|

On a graphing calculator, the two-proportion z-test function takes x₁, n₁, x₂, n₂ and the direction of Hₐ. It returns z, the p-value, p̂₁, p̂₂ and p̂c. Still write the formula with your numbers in it, so that your method is visible.

Interpreting the p-value

A p-value interpretation has three ingredients:

1. The assumption: “Assuming the true proportions are equal…”, written in context (for example, “assuming the true proportions of members who borrowed an e-book are the same in both towns”).
2. The probability: “…there is a [p-value] probability of getting…”
3. The event, in the direction of Hₐ: “…a difference in sample proportions (group 1 minus group 2) of [observed difference] or more” (use “or less” for a left-tailed test, and “at least as far from 0 in either direction” for a two-sided test).

The p-value is not the probability that H₀ is true, and it is not the probability that the result “happened by chance” with no assumption stated. It is a probability about the data, calculated assuming H₀.

Decision and conclusion

The formal decision compares the p-value with the significance level α, which was chosen before the data were collected:

• p-value ≤ α: reject H₀. There is convincing statistical evidence for Hₐ.
• p-value > α: fail to reject H₀. There is not convincing statistical evidence for Hₐ.

A full conclusion has three parts: the linkage (compare the p-value with α, using both numbers), the decision, and a context sentence in terms of Hₐ. The context sentence refers to the parameters (the true proportions, or their difference) and to the populations or treatments, and uses non-definitive language such as “there is convincing evidence that”. Never write “accept H₀” or “this proves”.

Then use the result to answer the investigative question that started the study. What you may claim depends on how the data were collected:

• Independent random samples let you generalise to the two populations sampled.
• A randomized experiment lets you conclude that the difference in treatments caused the difference in responses, for subjects like those in the experiment.

Worked example 1: independent random samples, one-sided test

Context (fictional). The Westmoor library service runs libraries in two towns, Ashby Vale (about 6,500 adult members) and Corran (about 9,000 adult members). The investigative question is: Is the proportion of adult members who borrowed an e-book in the past month greater in Ashby Vale than in Corran? The service selects independent random samples of 180 Ashby Vale members and 220 Corran members. In Ashby Vale, 81 had borrowed an e-book in the past month; in Corran, 77 had. Use α = 0.05.

1. Hypotheses. Let p₁ = the true proportion of all Ashby Vale adult members who borrowed an e-book in the past month, and p₂ = the true proportion of all Corran adult members who did.

• H₀: p₁ − p₂ = 0
• Hₐ: p₁ − p₂ > 0
• α = 0.05

2. Method and conditions. Two-sample z-test for the difference between two population proportions.

• Randomization: independent random samples from the two towns’ members.
• 10% condition: 180 ≤ 10% of 6,500 = 650, and 220 ≤ 10% of 9,000 = 900.
• Normality: p̂c = (81 + 77) ÷ (180 + 220) = 158 ÷ 400 = 0.395. Then n₁p̂c = 71.1, n₁(1 − p̂c) = 108.9, n₂p̂c = 86.9 and n₂(1 − p̂c) = 133.1, all at least 10.

3. Calculations.

• p̂₁ = 81 ÷ 180 = 0.45 and p̂₂ = 77 ÷ 220 = 0.35, so p̂₁ − p̂₂ = 0.10
• Standard error: √[0.395 × 0.605 × (1/180 + 1/220)] ≈ 0.04913
• z = (0.10 − 0) ÷ 0.04913 ≈ 2.04
• p-value = P(Z ≥ 2.04) ≈ 0.0209 (right tail, because Hₐ uses >). A table with z = 2.04 gives 0.0207.

Interpretation of the p-value. Assuming the true proportions of adult members who borrowed an e-book in the past month are the same in Ashby Vale and Corran, there is about a 0.0209 probability of getting a difference in sample proportions (Ashby Vale minus Corran) of 0.10 or more by chance in random sampling.

4. Conclusion. Because the p-value of 0.0209 is less than α = 0.05, we reject H₀. There is convincing statistical evidence that the true proportion of adult members who borrowed an e-book in the past month is greater in Ashby Vale than in Corran (p₁ − p₂ > 0).

Answer to the investigative question. Yes: the samples give convincing evidence that a greater proportion of Ashby Vale members borrowed an e-book in the past month.

Check. The decision depends on α. With α = 0.01, chosen in advance, the same p-value (0.0209 > 0.01) would lead you to fail to reject H₀. This is why α must be fixed before you see the data.

Worked example 2: a randomized experiment, two-sided test

Context (fictional). A bike-share company is testing two versions of its sign-up screen. During one week, 300 people who downloaded the app were randomly assigned, 150 to each version. With Version A, 96 completed sign-up; with Version B, 84 did. The investigative question is: Does the sign-up screen version affect the proportion of new users who complete sign-up? Use α = 0.05.

1. Hypotheses. Let p₁ = the true proportion of new users like these who would complete sign-up with Version A, and p₂ = the true proportion who would complete sign-up with Version B.

• H₀: p₁ = p₂
• Hₐ: p₁ ≠ p₂ (the question asks whether the version affects sign-up, with no direction)
• α = 0.05

2. Method and conditions. Two-sample z-test for the difference between two population proportions.

• Randomization: users were randomly assigned to the two versions (a randomized experiment).
• 10% condition: not needed, because the data come from a randomized experiment, not from sampling a population.
• Normality: p̂c = (96 + 84) ÷ 300 = 0.60, so n₁p̂c = n₂p̂c = 90 and n₁(1 − p̂c) = n₂(1 − p̂c) = 60, all at least 10.

3. Calculations.

• p̂₁ = 96 ÷ 150 = 0.64 and p̂₂ = 84 ÷ 150 = 0.56, so p̂₁ − p̂₂ = 0.08
• Standard error: √[0.60 × 0.40 × (1/150 + 1/150)] = √0.0032 ≈ 0.05657
• z = 0.08 ÷ 0.05657 ≈ 1.41
• p-value = 2 × P(Z ≥ 1.41) ≈ 0.1573 (technology, unrounded z). A table with z = 1.41 gives 2 × 0.0793 = 0.1586.

Interpretation of the p-value. Assuming the version of the sign-up screen makes no difference to the true proportion who complete sign-up, there is about a 0.1573 probability of getting a difference in sample proportions at least as far from 0 as 0.08, in either direction, by chance in the random assignment.

4. Conclusion. Because the p-value of 0.1573 is greater than α = 0.05, we fail to reject H₀. There is not convincing statistical evidence that the true proportion of new users like these who complete sign-up differs between Version A and Version B.

Answer to the investigative question. The experiment does not give convincing evidence that the screen version affects sign-up. This does not show that the two versions are equally effective; an 8-percentage-point difference this size happens quite often by chance with groups of 150.

Check. If you had wrongly used a one-sided test, you would have reported 0.0786, half the correct value. The direction of Hₐ comes from the question, never from the data.

Common misconceptions

• Averaging p̂₁ and p̂₂ to get p̂c. Add the successes and add the sample sizes. The simple average is only correct when n₁ = n₂.
• Using the unpooled standard error in the test. √[p̂₁(1 − p̂₁)/n₁ + p̂₂(1 − p̂₂)/n₂] belongs to the confidence interval. The test assumes p₁ = p₂, so it uses p̂c.
• Choosing the tail from the sign of z. The tail comes from Hₐ. If Hₐ is p₁ > p₂ but z is negative, the p-value is greater than 0.5.
• Forgetting to double for a two-sided test.
• “The p-value is the probability that the two proportions are equal.” It is the probability of a result at least this extreme, assuming they are equal.
• “We accept H₀, so the proportions are the same.” Failing to reject is a lack of evidence, not proof of equality.
• Concluding about p̂₁ and p̂₂. You already know the sample proportions differ. The conclusion is about the true proportions in the populations or treatments.
• Claiming cause from independent samples. Only a randomized experiment supports a cause-and-effect conclusion.

Where this leads

So far you have compared one proportion across two groups. Next, in Topic 3.14, Setting Up a Chi-Square Test for Homogeneity or Independence, you will compare whole distributions of a categorical variable, across two or more groups, using a new test statistic. To revise the set-up for this test, go back to Topic 3.12. Try the practice questions now, then use the revision notes and the checklist to consolidate.