Where you are in the test
In Topic 4.9 you set up a two-sample t-test: you defined μ₁ and μ₂, wrote the hypotheses and checked the conditions. That was State and Plan. This topic finishes the job with Do (test statistic and p-value) and Conclude (decision and conclusion in context).
The logic is the same as every test you have met. Assume H₀ is true, so the two population means are equal and μ₁ − μ₂ = 0. Ask how surprising your observed difference x̄₁ − x̄₂ would be in that world. If it would be very surprising, you have evidence against H₀.
The test statistic
The two-sample t-statistic measures how far the observed difference is from the hypothesised difference of 0, in standard errors:
t = ((x̄₁ − x̄₂) − 0) / √(s₁²/n₁ + s₂²/n₂)
- The numerator is the observed difference minus the null value 0.
- The denominator is the standard error of x̄₁ − x̄₂, the same one you used for the interval in Topic 4.7. Square each s, divide by its own n, add, then take one square root.
A t of 2.5 means the sample means differ by 2.5 standard errors. A t near 0 means the samples are close to what H₀ predicts.
Keep the order of subtraction from your hypotheses. If Hₐ is μ_E − μ_B > 0, calculate x̄_E − x̄_B. Reversing the order flips the sign of t, and you could end up shading the wrong tail.
Degrees of freedom and the p-value
When H₀ is true and the conditions hold, the t-statistic has approximately a t-distribution. Its degrees of freedom (df) come from a long formula that technology applies for you. The result:
smaller of (n₁ − 1, n₂ − 1) ≤ df ≤ n₁ + n₂ − 2
The p-value is the area in the tail (or tails) of that t-distribution, beyond the observed t, in the direction of Hₐ:
| Hₐ | p-value |
|---|---|
| μ₁ − μ₂ > 0 | P(t ≥ observed t), the upper tail |
| μ₁ − μ₂ < 0 | P(t ≤ observed t), the lower tail |
| μ₁ − μ₂ ≠ 0 | 2 × P(t ≥ |observed t|), both tails |
Two ways to find it:
- Technology (preferred). A graphing calculator’s two-sample t-test gives t, df and the p-value together. Choose not to pool: the course does not assume the population standard deviations are equal.
- A t-table. Use the conservative df, the smaller of n₁ − 1 and n₂ − 1. If that row is not printed, use the next lower df in the table. Find the two critical values that the observed t lies between and read off a range for the p-value. A smaller df gives a slightly larger p-value, so this never overstates your evidence.
Interpreting the p-value
The p-value is a conditional probability. It is calculated assuming H₀ is true, and an interpretation must say so. A reliable template:
Assuming the true mean [response] of [population 1] and [population 2] are equal, there is a [p-value] probability of getting a difference in sample means (1 minus 2) of [observed difference] or [more extreme, in the direction of Hₐ] by chance alone.
For a two-sided test, “more extreme” means “at least as far from 0 in either direction”.
The p-value is not the probability that H₀ is true, and it is not the probability that the result happened by chance. It describes how unusual your data would be if the means were equal.
Decision and conclusion
Formal decision. Compare the p-value with the significance level α, which was chosen before the data were seen:
- p-value ≤ α: reject H₀.
- p-value > α: fail to reject H₀.
Write the comparison explicitly, for example “Because 0.012 ≤ 0.05, we reject H₀.”
Conclusion in context. The conclusion is about Hₐ, uses non-definitive language and names the parameters and populations:
- Reject H₀: “There is convincing evidence that the mean [response] of all [population 1] is greater than (less than, different from) that of all [population 2].”
- Fail to reject H₀: “There is not convincing evidence that the mean [response] of all [population 1] is greater than (less than, different from) that of all [population 2].”
Never write “proves”, “accept H₀” or “the means are equal”.
Answering the investigative question. A study begins with a question in plain words, such as “Do students at the later-starting school sleep longer?” The test result is the statistical reasoning behind your answer. Finish with a sentence that answers that question directly, and say what the design allows: random samples let you generalise to the populations sampled; random assignment lets you talk about cause and effect.
Worked example 1: two random samples, one-sided
Context. Two fictional secondary schools: Brookfield (1,400 students) starts at 8:00 and Elmstead (1,650 students) starts at 8:45. A researcher believes that Elmstead students sleep longer on school nights. She selects a random sample of 48 Brookfield students and, separately, 52 Elmstead students. Each keeps a sleep diary for one school week, and she records each student’s mean nightly sleep in hours.
| School | n | x̄ (hours) | s (hours) |
|---|---|---|---|
| Elmstead | 52 | 7.62 | 0.84 |
| Brookfield | 48 | 7.21 | 0.93 |
Question. Do the data give convincing evidence, at α = 0.05, that Elmstead students sleep longer on average?
State. μ_E and μ_B = the mean nightly school-night sleep (hours) of all students at Elmstead and at Brookfield. H₀: μ_E − μ_B = 0. Hₐ: μ_E − μ_B > 0.
Plan. Two-sample t-test for μ_E − μ_B. Randomization: two independent random samples ✓. 10%: 52 ≤ 165 and 48 ≤ 140 ✓. Sample data: 52 ≥ 30 and 48 ≥ 30 ✓.
Do.
- Observed difference: 7.62 − 7.21 = 0.41 hours.
- Variance terms: 0.84²/52 = 0.013569 and 0.93²/48 = 0.018019. Sum = 0.031588.
- SE = √0.031588 = 0.17773 hours.
- t = (0.41 − 0) / 0.17773 = 2.31.
- Technology: df = 94.86 (between 47 and 98, as expected). p-value = P(t ≥ 2.31) = 0.0116.
Interpret the p-value. Assuming the mean nightly sleep of all Elmstead students and all Brookfield students is the same, there is about a 0.0116 probability of getting a difference in sample means (Elmstead minus Brookfield) of 0.41 hours or more by random sampling alone.
Conclude. Because 0.0116 ≤ 0.05, we reject H₀. There is convincing evidence that the mean school-night sleep of all Elmstead students is greater than that of all Brookfield students.
Answer the question. Yes: the data support the researcher’s belief that Elmstead students sleep longer on average. But this is an observational study. The schools differ in more than start time (catchment, homework policy, travel), so we cannot conclude that the later start causes the extra sleep.
Check with a table. Conservative df = 48 − 1 = 47. Using the df = 40 row, the one-tail critical values are 2.021 (tail 0.025) and 2.423 (tail 0.01). Since 2.021 < 2.31 < 2.423, 0.01 < p-value < 0.025. Same decision. Note that at α = 0.01 the decision would change (0.0116 > 0.01), which is why α must be fixed before you look at the data.
Worked example 2: a randomized experiment, two-sided
Context. A fictional psychology class asks whether background music affects memory. Twenty-four volunteers were randomly assigned, 12 to study a list of 30 words for five minutes with instrumental music playing and 12 to study the same list in silence. Each then wrote down as many words as they could remember.
- Music: 14, 18, 12, 16, 15, 19, 13, 17, 15, 11, 16, 20
- Silence: 16, 19, 15, 18, 13, 21, 17, 14, 18, 16, 20, 15
Question. Carry out a test at α = 0.05.
State. μ_M and μ_S = the true mean number of words recalled by volunteers like these who study with music and who study in silence. H₀: μ_M − μ_S = 0. Hₐ: μ_M − μ_S ≠ 0.
Plan. Two-sample t-test for μ_M − μ_S. Randomization: conditions randomly assigned ✓. 10%: not needed (experiment). Sample data: both groups have 12 < 30, so check both. Music: Q1 = 13.5, Q3 = 17.5, fences 7.5 and 23.5. Silence: Q1 = 15, Q3 = 18.5, fences 9.75 and 23.75. All values lie inside the fences and neither sample is strongly skewed ✓.
Do.
- Means: x̄_M = 186 ÷ 12 = 15.5 words; x̄_S = 202 ÷ 12 = 16.83 words. Difference = −1.33 words.
- Standard deviations (Sx): s_M = 2.7469 and s_S = 2.4433 words.
- SE = √(2.7469²/12 + 2.4433²/12) = √(0.628788 + 0.497475) = √1.126263 = 1.0613 words.
- t = (−1.3333 − 0) / 1.0613 = −1.26.
- Technology: df = 21.70. p-value = 2 × P(t ≤ −1.26) = 0.222.
Interpret the p-value. Assuming the true mean recall is the same with music and in silence, there is about a 0.222 probability that random assignment alone would produce a difference in group means of 1.33 words or more, in either direction.
Conclude. Because 0.222 > 0.05, we fail to reject H₀. There is not convincing evidence that the true mean number of words recalled differs between studying with music and studying in silence, for volunteers like these.
Answer the question. These data do not show that background music affects memory. They do not show that it has no effect: a small effect could exist that 24 volunteers could not detect.
Check against an interval. The 95% two-sample t-interval for μ_M − μ_S is (−3.54, 0.87) words. It contains 0, which matches the decision. A two-sided test at α and a (1 − α) confidence interval calculated with the same df always agree in this way, as Topic 4.8 suggested.
Common misconceptions
- “The p-value is the probability that H₀ is true.” It is the probability of data like yours, or more extreme, assuming H₀ is true.
- “We accept H₀” or “the means are equal.” Failing to reject H₀ only means the evidence was not convincing.
- “The test proves the means differ.” Use “convincing evidence”; samples can mislead.
- Forgetting to double for a two-sided test, or doubling for a one-sided test.
- Using df = n₁ + n₂ − 2. That is the upper limit and belongs to the pooled test, which the course does not use. Use technology or the conservative smaller n − 1.
- Adding standard deviations in the denominator instead of variance terms.
- Comparing t with α. Compare the p-value with α.
- A conclusion about H₀ or the samples. Write it about Hₐ and the population means, in context.
- Claiming cause and effect from two random samples. Only random assignment supports a causal conclusion.
Where this leads
This completes inference for means. Unit 5, Regression Analysis, turns to relationships between two quantitative variables, starting with Topic 5.1, Graphical Representations Between Two Quantitative Variables. First, try the practice questions, then use the revision notes and the checklist to consolidate. To revise the set-up, go back to Topic 4.9.
Quick questions
Why are my degrees of freedom not a whole number?
The two-sample t-test uses an approximation for the degrees of freedom that depends on both sample sizes and both standard deviations. The result is usually a decimal, and it always lies between the smaller of n₁ − 1 and n₂ − 1 and n₁ + n₂ − 2.
What if I cannot use technology for the p-value?
Use the conservative degrees of freedom, the smaller of n₁ − 1 and n₂ − 1, and a t-table. The table gives a range for the p-value, for example between 0.01 and 0.025, which is usually enough to compare with α.
If I fail to reject H₀, have I shown the two means are equal?
No. You have only failed to find convincing evidence of a difference. A real difference might exist that your samples were too small or too variable to detect.