Residuals: Study Guide (Statistics 5.4)

How wrong was the prediction?

In Topic 5.3 you used a linear regression model ŷ = a + bx to make predictions. Real data points do not sit exactly on the line, so most predictions are a little too high or a little too low. A residual measures that error for one data point:

residual = y − ŷ = observed value − predicted value

• The residual has the same units as the response variable.
• On a scatterplot, it is the vertical distance from the point to the line: up is positive, down is negative.

What the sign tells you

Residual	Point is…	The model…	In words
positive (y > ŷ)	above the line	underpredicts (underestimates)	the actual value was bigger than predicted
negative (y < ŷ)	below the line	overpredicts (overestimates)	the actual value was smaller than predicted
zero	on the line	predicts exactly	the prediction was exactly right for this point

A good memory aid: residual = actual − predicted. If the actual value is bigger, the answer is positive and the model guessed too low.

The data set used in this guide

A fictional taxi company in the city of Port Avery recorded the distance (km) and the fare ($) for 10 rides. Fares depend mainly on distance, but waiting in traffic adds extra charges, so the points scatter around a line.

Distance (km)	2.1	3.5	4.0	5.2	6.8	7.5	8.3	9.9	11.2	12.6
Fare ($)	8.40	8.80	12.30	12.80	14.40	18.40	18.00	23.20	23.10	26.80

Technology gives the model ŷ = 3.83 + 1.80x, where x is the distance in km and ŷ is the predicted fare in dollars.

Worked example 1: calculating and interpreting residuals

Question. (a) Find and interpret the residual for the 9.9 km ride. (b) Find and interpret the residual for the 6.8 km ride. (c) Which of the 10 rides did the model predict least well?

(a)

1. Predicted fare: ŷ = 3.83 + 1.80(9.9) = 3.83 + 17.82 = $21.65.
2. Residual = observed − predicted = 23.20 − 21.65 = +$1.55.
3. Interpretation: the actual fare for the 9.9 km ride was $1.55 more than the model predicted. The model underpredicted this fare.

(b)

1. ŷ = 3.83 + 1.80(6.8) = 3.83 + 12.24 = $16.07.
2. Residual = 14.40 − 16.07 = −$1.67.
3. Interpretation: the actual fare was $1.67 less than predicted. The model overpredicted this fare.

(c) The full set of residuals, in order of distance, is:

Distance (km)	2.1	3.5	4.0	5.2	6.8	7.5	8.3	9.9	11.2	12.6
ŷ ($)	7.61	10.13	11.03	13.19	16.07	17.33	18.77	21.65	23.99	26.51
Residual ($)	0.79	−1.33	1.27	−0.39	−1.67	1.07	−0.77	1.55	−0.89	0.29

The residual furthest from 0 is −1.67, for the 6.8 km ride. “Least well” means the largest residual in size, ignoring the sign.

Check. The residuals add to −0.08, which is very close to 0. For a least-squares line the residuals always add to exactly 0 (Topic 5.5); the small difference here comes from rounding a and b.

Worked example 2: working backwards from a residual

A residual question can also run the other way: you know the residual and must find the observed value.

observed y = ŷ + residual

Question. (a) A 10.5 km ride had a residual of −$0.86. What was the actual fare? (b) A 6.0 km ride cost $15.80. Find its residual and say whether the model under- or overpredicted.

(a)

1. Predicted fare: ŷ = 3.83 + 1.80(10.5) = 3.83 + 18.90 = $22.73.
2. Observed fare = ŷ + residual = 22.73 + (−0.86) = $21.87.
3. The negative residual means the actual fare was less than predicted, and $21.87 is indeed less than $22.73.

(b)

1. ŷ = 3.83 + 1.80(6.0) = 3.83 + 10.80 = $14.63.
2. Residual = 15.80 − 14.63 = +$1.17.
3. The model underpredicted this fare by $1.17. Perhaps the taxi waited in traffic.

Writing a residual interpretation

A full interpretation of a residual has four parts. Use this frame and fill in the context:

“The actual [response, with units] for [this individual, with its x-value] was [size of residual] more / less than the model predicted, so the model underpredicted / overpredicted.”

For example: “The actual fare for the 9.9 km ride was $1.55 more than the model predicted, so the model underpredicted it.”

Check each part before you move on:

• Size: give the residual without its sign; the words “more” or “less” carry the direction.
• Direction: positive means more than predicted; negative means less.
• Context: name the response variable and the individual, not just “y”.
• Units: the residual is in the units of the response.

Residual plots

A residual plot is a scatterplot of the residuals (vertical axis) against the explanatory variable x or the predicted values ŷ (horizontal axis). A horizontal line at residual = 0 marks where the model is exactly right.

A residual plot takes away the overall upward or downward trend, so any pattern that the line has missed becomes easy to see. You use it to judge whether a linear model is appropriate.

What you see in the residual plot	What it means
Random scatter above and below 0, with no clear pattern	The form is linear. A linear model is appropriate.
Curvature: a U-shape or an arch (residuals positive, then negative, then positive, or the reverse)	The form is not linear. A linear model is not the most appropriate model.

With one explanatory variable, a plot against ŷ shows the same pattern as a plot against x (mirrored left to right if the slope b is negative), so either is fine.

Worked example 3: a high r, but a curved residual plot

Question. A student charged a phone of a fictional model from 5% and recorded the battery level every 10 minutes.

Time (min)	0	10	20	30	40	50	60	70	80	90
Battery (%)	5	24	39	51	61	69	75	80	84	87

Technology gives r = 0.96 and ŷ = 18.15 + 0.874x. The residual plot is plot B in Figure 2. Is a linear model appropriate?

1. Calculate a few residuals to see the pattern. At 0 min: ŷ = 18.15, residual = 5 − 18.15 = −13.15. At 40 min: ŷ = 18.15 + 0.874(40) = 53.11, residual = 61 − 53.11 = +7.89. At 90 min: ŷ = 18.15 + 0.874(90) = 96.81, residual = 87 − 96.81 = −9.81.
2. Describe the residual plot. The residuals are negative at the start, positive in the middle, and negative again at the end. They form a clear arch, not random scatter.
3. Link to the data. The battery gains 19 percentage points in the first 10 minutes but only 3 in the last 10. The charging slows down, so the true relationship curves. A straight line is too high at both ends and too low in the middle.
4. Conclusion. Despite r = 0.96, the curvature in the residual plot shows that a linear model is not the most appropriate model for these data. A curved model would fit better.

Check. The model also predicts 123.03% after 120 minutes, which is impossible for a battery. That extrapolation fails for the same reason: the line ignores the levelling off.

Common misconceptions

• Residual = predicted − observed. It is observed − predicted, y − ŷ. Reversing the order flips the sign and the interpretation.
• “A negative residual means the model underpredicted.” Negative means the actual value was below the prediction, so the model overpredicted.
• Forgetting units. A residual is in the units of the response: dollars, kg, minutes.
• “Least well predicted” means the most negative residual. It means the residual furthest from 0 in either direction.
• “r is close to 1, so the residual plot doesn’t matter.” Worked example 3 has r = 0.96 and a clearly curved residual plot.
• “Random scatter means the predictions are exact.” Random scatter shows the form is linear. The residuals can still be large if the association is weak.
• Plotting residuals against y. Plot them against x or against ŷ, not against the observed y-values.
• “A residual of 0 means the model is perfect.” It means that one point lies on the line. Other points can still be far away.

Where this leads

The least-squares regression line is the line that makes the sum of the squared residuals as small as possible. In Topic 5.5: Least-Squares Regression you will find its slope and intercept with technology, interpret them in context, and meet r², the coefficient of determination. Try the practice questions now, then use the revision notes and the checklist to consolidate.