Linear Regression Models: Study Guide (Statistics 5.3)

From a scatterplot to a model

In Topic 5.1 you described a scatterplot. In Topic 5.2 you used r to measure how closely the points follow a straight line. Now you take the next step: you draw a line through the points and use it to predict.

A linear regression model is a straight-line equation that uses one variable to predict another.

• The explanatory variable, x, is the one you use to make the prediction. It goes on the horizontal axis.
• The response variable, y, is the one you want to predict. It goes on the vertical axis.

A line is only a sensible model when the form of the scatterplot looks linear. If the points follow a curve, a straight line will predict badly in places, however strong the association looks. Always look at the scatterplot first.

The equation ŷ = a + bx

A linear regression model is written

ŷ = a + bx

Symbol	Name	Meaning
ŷ	“y-hat”	the predicted value of the response variable
x	explanatory variable	the value you substitute in
a	y-intercept	the value of ŷ when x = 0
b	slope	the change in ŷ for each increase of 1 in x

The hat matters. y is a real, observed value. ŷ is what the model predicts. They are usually different, because real points do not lie exactly on the line. Writing “y = a + bx” for a model suggests that every point lies on the line, which is wrong.

In context, replace the letters with the names of the variables. For example: predicted sales = −30.26 + 5.37 × (temperature). This makes clear what is being predicted and from what.

Where do a and b come from? In this topic they are given. In Topic 5.5 you will find them with technology, using the least-squares method, and you will learn to interpret the slope and intercept in context.

The data set used in this guide

A fictional kiosk on a beach, Shellbay Snacks, recorded the maximum temperature (°C) and the number of iced drinks sold on 12 summer days.

Max temperature (°C)	14	16	17	19	20	22	23	25	26	28	29	31
Iced drinks sold	44	60	55	71	84	79	99	96	119	112	136	131

The scatterplot shows a strong, positive, linear association with no unusual points (r = 0.97). Temperature is the explanatory variable, because the owner wants to use the weather forecast to predict sales. Technology gives the model

ŷ = −30.26 + 5.37x

where x is the maximum temperature in °C and ŷ is the predicted number of iced drinks sold.

Interpolation and extrapolation

The x-values in the data run from 14 °C to 31 °C. This interval decides how far you can trust a prediction.

• Interpolation: predicting for an x-value inside the interval of x-values used to fit the line (here, from 14 to 31 °C, end points included). You have evidence that the linear pattern holds there.
• Extrapolation: predicting for an x-value outside that interval (here, below 14 °C or above 31 °C). You have no data there, so you are assuming the straight-line pattern continues.

Extrapolation is less reliable, and the further outside the interval you go, the less reliable it becomes. A small step outside may give a reasonable estimate. A large step can give nonsense, such as a negative number of drinks.

Worked example 1: making and judging predictions

Question. Use the model ŷ = −30.26 + 5.37x for the Shellbay Snacks kiosk. (a) Predict the number of iced drinks sold on a day with a maximum temperature of 24 °C. (b) Predict sales for 38 °C and for 4 °C. (c) For each prediction, say whether it is an interpolation or an extrapolation, and how far you would trust it.

(a)

1. Substitute x = 24: ŷ = −30.26 + 5.37(24).
2. Multiply first: 5.37 × 24 = 128.88.
3. Add: ŷ = −30.26 + 128.88 = 98.62.
4. In context: the model predicts that the kiosk will sell about 99 iced drinks on a day with a maximum temperature of 24 °C.

(b)

• x = 38: ŷ = −30.26 + 5.37(38) = −30.26 + 204.06 = 173.80, about 174 drinks.
• x = 4: ŷ = −30.26 + 5.37(4) = −30.26 + 21.48 = −8.78 drinks.

(c)

Temperature	ŷ	Inside 14 to 31 °C?	Type	Trust?
24 °C	98.62	yes	interpolation	reasonable: the data show a linear pattern here
38 °C	173.80	no, 7 °C above	extrapolation	doubtful: we have no days this hot. Sales might level off (the kiosk could run out of ice, or people might stay away from the beach).
4 °C	−8.78	no, 10 °C below	extrapolation	not believable: a negative number of drinks is impossible

Check. The answer to (a) should lie between the values seen at nearby temperatures. At 23 °C the kiosk sold 99 drinks and at 25 °C it sold 96, so a prediction of about 99 at 24 °C is sensible. Notice also that the prediction for (a) is a predicted value. On a real 24 °C day the kiosk will probably sell a few more or a few fewer.

Reading the slope and intercept (a first look)

You can read two things straight from the equation:

• Slope, b = 5.37. For each extra 1 °C, the predicted number of drinks goes up by 5.37. You can check this: ŷ at 25 °C minus ŷ at 24 °C is 5.37.
• y-intercept, a = −30.26. This is ŷ when x = 0 °C. Here x = 0 is far outside the data, so a = −30.26 is an extrapolation and has no sensible meaning: you cannot sell −30 drinks.

You will interpret the slope and intercept formally, in context, in Topic 5.5.

Worked example 2: a growing sunflower

Question. A student measured the height of a sunflower of a fictional variety on eight days, from day 12 to day 40 after sowing. The scatterplot of height against days since sowing is strongly linear (r = 0.99). Technology gives

ŷ = −22.12 + 3.59x

where x is the number of days since sowing and ŷ is the predicted height in centimetres.

(a) Predict the height on day 30. (b) Predict the height on day 44 and on day 365. Compare how reliable the two predictions are. (c) The model predicts a height of −4.17 cm on day 5. What does this tell you?

(a) ŷ = −22.12 + 3.59(30) = −22.12 + 107.70 = 85.58 cm. Day 30 lies inside the interval 12 to 40, so this is an interpolation. The model predicts the sunflower is about 85.6 cm tall 30 days after sowing.

(b)

1. Day 44: ŷ = −22.12 + 3.59(44) = −22.12 + 157.96 = 135.84 cm.
2. Day 365: ŷ = −22.12 + 3.59(365) = −22.12 + 1,310.35 = 1,288.23 cm, about 12.9 metres.
3. Both are extrapolations, because both are after day 40.
4. Day 44 is only 4 days beyond the data. It is less reliable than an interpolation, but the plant may well still be growing at a similar rate, so the estimate could be reasonable.
5. Day 365 is 325 days beyond the data. The model assumes the plant grows by about 3.59 cm every day for a whole year. Real plants stop growing (and sunflowers die after one season), so 12.9 metres is not believable. The further you extrapolate, the less reliable the prediction.

(c) Day 5 is before the first measurement (day 12), so this is also an extrapolation. A negative height is impossible. The straight-line pattern seen between days 12 and 40 does not describe the early days, when the seedling was just emerging. This shows that extrapolation can fail in both directions.

Check. Over 10 days the model predicts growth of 10 × 3.59 = 35.9 cm, which fits the data: the measured heights rose from 20 cm on day 12 to 125 cm on day 40.

A checklist for any prediction

1. Is a line appropriate? The scatterplot should show a linear form.
2. Which variable is which? Substitute the explanatory variable to predict the response.
3. Substitute and calculate. Multiply b by x, then add a. Keep the coefficients as given; round only at the end.
4. Report in context. Say “predicted”, name the variable and give the units.
5. Interpolation or extrapolation? Compare x with the smallest and largest x-values in the data, and comment on reliability.

Common misconceptions

• Writing y instead of ŷ. “y = −30.26 + 5.37x” claims every point is on the line. The model gives predicted values, ŷ.
• “The prediction is what will happen.” ŷ is an estimate of a typical value for that x. Individual days vary around it.
• “Extrapolation is always wrong” or “always fine”. Extrapolation is not banned, but it is less reliable, and more so the further you go. A step just outside the data is safer than a huge one.
• “A prediction at the largest x-value is an extrapolation.” The interval includes its end points, so it is an interpolation.
• Substituting the wrong variable. The model predicts the response from the explanatory variable. Do not put a y-value into the place of x.
• Mixing up a and b. In ŷ = a + bx, b multiplies x. In ŷ = −30.26 + 5.37x, the slope is 5.37, not −30.26.
• Using a line for a curved pattern. A high r does not prove the form is linear. Check the scatterplot (and, in Topic 5.4, the residual plot).
• Rounding too early. Rounding 5.37 to 5 before predicting at 24 °C gives 89.74 instead of 98.62, an error of almost 9 drinks.
• No context. “ŷ = 98.62” alone is incomplete. Say “the model predicts about 99 iced drinks on a 24 °C day”.

Where this leads

A prediction ŷ is rarely exactly right. The difference between what really happened and what the model predicted is called a residual, and residuals help you judge whether a line is a good model. That is Topic 5.4: Residuals. Try the practice questions now, then use the revision notes and the checklist to consolidate.