Setting Up a Chi-Square Test for Homogeneity or Independence: Study Guide (Statistics 3.14)

Comparing whole distributions

In Topics 3.12 and 3.13 you compared one proportion in two groups: the proportion of “successes”. Many questions are bigger than that. A categorical variable often has three or more categories, and you may want to compare three or more groups. For example:

• Do students at three colleges differ in where they mainly study (library, home or somewhere else)?
• In one city, is the age group of an adult associated with their main source of news?

A single two-sample z-test cannot answer these. You need a test that compares whole distributions in a two-way table. That is a chi-square test (χ², “kye-square”). There are two versions:

• the chi-square test for homogeneity, and
• the chi-square test for independence.

The calculations are the same for both. This topic is about setting up the test: choosing the version, writing hypotheses and checking conditions. Topic 3.15 carries it out.

What the chi-square statistic measures

Every chi-square test compares two sets of counts in the two-way table:

• Observed counts: what the sample actually produced.
• Expected counts: what you would expect in each cell if H₀ were true.

The chi-square statistic measures how far the observed counts are from the expected counts, relative to the expected counts. For each cell, the gap between observed and expected is squared (so it cannot be negative) and divided by the expected count (so a gap of 6 counts for more when 10 were expected than when 100 were). The cell values are then added.

This leads to three facts you can use now:

1. χ² is never negative.
2. χ² = 0 only if every observed count equals its expected count exactly.
3. The larger χ², the further the data are from what H₀ predicts, so the stronger the evidence against H₀.

Preview: the formula, χ² = Σ (observed − expected)² / expected, and the full calculation are taught in Topic 3.15.

Chi-square distributions

If H₀ is true, the chi-square statistic follows (approximately) a chi-square distribution. These form a family of density curves. Each member is identified by its degrees of freedom (df), which depend on the number of rows and columns in the table (Topic 3.15 shows how).

All chi-square distributions:

• take only positive values (they start at 0 and have no left tail below it);
• are skewed right, with a long tail to the right;
• become less skewed as the degrees of freedom increase, and their peak moves to the right.

Because large values of χ² are the evidence against H₀, the p-value for a chi-square test is always the area in the right tail.

Homogeneity or independence?

Decide by asking how the data were collected.

	Chi-square test for homogeneity	Chi-square test for independence
Design	Independent random samples from two or more populations, or two or more treatment groups in a randomized experiment	One random sample from one population
Variables	One categorical variable, recorded for each sample or group	Two categorical variables, both recorded for each individual
Question	Is the distribution of the variable different across the populations or treatments?	Is there an association between the two variables in the population?
The groups	fixed by the researcher before the data are collected	emerge from the data (the row totals are not chosen)

A quick test: if the researcher chose how many individuals to take from each group, it is usually homogeneity. If the researcher took one sample and sorted it by two variables afterwards, it is independence.

A test for homogeneity should name the categorical variable and the populations (or treatments). A test for independence should name both categorical variables and the one population.

Writing the hypotheses

Hypotheses for chi-square tests are written in words, in context, about populations, not about samples.

Homogeneity

• H₀: There is no difference in the distribution of [variable] across [populations or treatments].
• Hₐ: There is a difference in the distribution of [variable] across [populations or treatments].

Independence

• H₀: There is no association between [variable 1] and [variable 2] among [population]. (Equivalently: the two variables are independent in this population.)
• Hₐ: There is an association between [variable 1] and [variable 2] among [population]. (Equivalently: they are not independent.)

Hₐ does not say that every distribution is different from every other. One difference anywhere in the table is enough to make Hₐ true.

The three conditions

1. Randomization.
   • Homogeneity: the data come from independent random samples, or from a randomized experiment.
   • Independence: the data come from one random sample.
2. 10% condition. When sampling without replacement, each sample size should be no more than 10% of its population: n ≤ 10%N. This is not needed for a randomized experiment.
3. Expected counts condition. All expected counts should be greater than 5.

To check condition 3 you need the expected counts. Technology produces them. Preview of Topic 3.15: each expected count is (row total × column total) ÷ table total. You will practise this calculation in the next topic; here you mainly need to compare the expected counts with 5.

If a condition is not met, say so and say what it means. If the randomization condition fails, you cannot generalise to a population. If the expected counts condition fails, the chi-square distribution is not a good model for the statistic, so the test should not be carried out as it stands. A larger sample would help. Sometimes it makes sense to combine categories, but this changes the question being asked, so say so.

Worked example 1: a test for homogeneity

Context (fictional). A researcher wants to know whether students at three colleges differ in where they mainly study. She takes independent random samples of 120 students from Elmfield College (1,800 students), 100 from Northcote College (1,500 students) and 80 from Riverside College (1,100 students). Each student names their main study place.

	Library	Home	Other	Total
Elmfield	48	52	20	120
Northcote	30	55	15	100
Riverside	22	44	14	80
Total	100	151	49	300

Question. Identify the appropriate test, state the hypotheses and check the conditions.

1. Test. There are three separate random samples, one from each college, and one categorical variable (main study place). This is a chi-square test for homogeneity.

2. Hypotheses.

• H₀: There is no difference in the distribution of main study place (library, home, other) among all students at Elmfield, Northcote and Riverside colleges.
• Hₐ: There is a difference in the distribution of main study place among all students at the three colleges.

3. Conditions.

• Randomization: independent random samples from the three colleges.
• 10% condition: 120 ≤ 10% of 1,800 = 180; 100 ≤ 10% of 1,500 = 150; 80 ≤ 10% of 1,100 = 110.
• Expected counts (from technology):

	Library	Home	Other
Elmfield	40.00	60.40	19.60
Northcote	33.33	50.33	16.33
Riverside	26.67	40.27	13.07

The smallest expected count is 13.07, so all expected counts are greater than 5. (Check one: Elmfield–Other = 120 × 49 ÷ 300 = 19.60.)

Conclusion of the set-up. All three conditions are met, so a chi-square test for homogeneity is appropriate. Topic 3.15 carries it out.

Check. The observed counts for Elmfield–Library (48) and Elmfield–Home (52) are some distance from their expected counts (40.00 and 60.40). Gaps like these make χ² larger. Whether they are large enough to be convincing is what the p-value will tell you.

Worked example 2: a test for independence where a condition fails

Context (fictional). A local newspaper in the city of Kestrelton (about 250,000 adults) wants to know whether age group is associated with main news source among the city’s adults. It takes a random sample of 160 adults and records each person’s age group and main news source.

	Online	TV	Print	Total
18–34	40	12	2	54
35–59	30	24	4	58
60 and over	14	28	6	48
Total	84	64	12	160

Question. Identify the appropriate test, state the hypotheses and check the conditions.

1. Test. One random sample from one population, with two categorical variables recorded for each adult. This is a chi-square test for independence. Notice that the newspaper did not choose 54, 58 and 48; the row totals came out of the sample.

2. Hypotheses.

• H₀: There is no association between age group and main news source among adults in Kestrelton.
• Hₐ: There is an association between age group and main news source among adults in Kestrelton.

3. Conditions.

• Randomization: one random sample of the city’s adults.
• 10% condition: 160 ≤ 10% of 250,000 = 25,000.
• Expected counts (from technology):

	Online	TV	Print
18–34	28.35	21.60	4.05
35–59	30.45	23.20	4.35
60 and over	25.20	19.20	3.60

None of the three expected counts in the Print column (4.05, 4.35 and 3.60) is greater than 5. The expected counts condition is not met, so a chi-square test for independence should not be carried out on this table.

What next? The newspaper could take a larger random sample. Alternatively, it could combine TV and Print into one category, “TV or print”. The expected counts would then be 25.65, 27.55 and 22.80 for that column, all greater than 5. But the question has changed: the test would now ask about online versus traditional sources, not about all three sources.

Check. Do not be fooled by the observed counts. The observed count of 2 is small, but it is the expected counts that matter, and here the whole Print column fails.

Common misconceptions

• Choosing the test from the shape of the table. Both tests use a two-way table. The design decides: separate samples or treatments means homogeneity; one sample with two variables means independence.
• Checking observed counts against 5. The condition is about expected counts.
• Hypotheses about the sample. “There is no association in the sample” is not a hypothesis; you can see the sample. Talk about the population.
• “Hₐ: all the distributions are different.” Hₐ says there is a difference somewhere.
• “An association means one variable causes the other.” A test for independence on an observational sample cannot show cause and effect.
• “χ² can be negative” or “a small χ² is strong evidence”. χ² is never negative, and large values are evidence against H₀.
• Using percentages in the table. Chi-square tests work with counts. Convert percentages back to counts first.
• Checking the 10% condition for an experiment. It is only needed when sampling without replacement from a population.

Where this leads

Next, in Topic 3.15, Carrying Out a Chi-Square Test for Homogeneity or Independence, you will calculate expected counts, the chi-square statistic, the degrees of freedom and the p-value, and write a conclusion. To compare with tests for one proportion in two groups, look back at Topic 3.13. Try the practice questions now, then use the revision notes and the checklist to consolidate.