Cambridge O Level Statistics: Surveys and questionnaire design (4040) – Revision Notes

Quick recall for section 1.4 of the Cambridge O Level Statistics syllabus (4040), for examination in 2025, 2026 and 2027: general ideas of surveys, including closed and open questions in questionnaires. Full explanations and worked examples are in the study guide. Test yourself with the practice questions. Course links: hub, printable checklist and free diagnostics. For sampling and bias see Data and Its Collection.

Key terms

Term	Meaning
Population	The whole group you want information about
Census	Every member of the population is asked
Sample survey	Only a selected part of the population is asked
Sampling frame	The list the sample is chosen from
Questionnaire	The written set of questions
Closed question	Respondent picks from fixed answers
Open question	Respondent answers in their own words
Coding	Sorting open answers into categories so they can be counted
Pilot survey	A small trial of the questionnaire before the main survey
Response rate	Usable replies ÷ number asked × 100
Non-response bias	Error caused when people who do not reply differ from those who do

Closed or open: the comparison

	Closed question	Open question
Answer format	Choose an option	Own words
Speed for respondent	Fast	Slower
Speed of analysis	Fast: tally straight away	Slow: must be coded first
Detail and reasons	Little	Rich
Unexpected answers	Cannot appear	Can appear
Comparing answers	Easy	Harder, depends on coding
Main risk	No option fits, researcher’s options may be wrong	Low completion, coding judgement

Memory line: closed gives counts, open gives reasons.

Types of closed question

• Yes/No
• Multiple choice (tick one, or tick all that apply)
• Rating scale (for example very satisfied to very dissatisfied)
• Banded numbers (for example hours per week: 0–2, 3–5, 6 or more)

Response options: the three tests

1. Exhaustive: every possible answer has a home. Add “Other (please state)” or “Don’t know”.
2. Mutually exclusive: no answer fits two options.
3. Sensible and balanced: a logical order, and the same number of positive and negative options on a rating scale.

Band check for whole numbers: the next band must start one unit above the previous band ends.

Faulty:    0–10   10–20   25–30
Problems:  10 is in two bands; 21 to 24 are in none
Corrected: 0–10   11–20   21–30

Method: coding open answers

1. Read through the answers first.
2. Choose categories that cover them, with no overlap.
3. Keep “Other” small. If it holds a large share, the categories need changing.
4. Decide a rule for answers that fit two categories (for example, code the first point made).
5. Tally the answers and write a frequency table.
6. Convert to percentages or pie chart angles if asked. Check that the percentages total 100% and the angles total 360°.

Reminder: angle = (frequency ÷ total) × 360°. For 18 of 72 responses: 18/72 × 360° = 90°.

Wording faults

Fault	Spot it by	Fix
Leading	Suggests the “right” answer	Use neutral wording and a balanced scale
Loaded	Emotive words such as “waste” or “terrible”	Replace with plain words
Double-barrelled	“and” joins two ideas	Split into two questions
Vague	“often”, “sometimes”, “regularly”	Use numbers or a fixed time period
Jargon or long	Respondent has to re-read	Short, plain words
Hypothetical	Asks about the future or a long time ago	Ask about recent behaviour
Overlapping or missing options	Gaps or double entries	Apply the three tests

Rewriting questions: three quick models

Original	Fault	Rewrite
“How good was the brilliant new gym?”	Loaded and leading	“How would you rate the new gym? Very good, good, neither, poor, very poor”
“Do you walk or cycle to school?”	Options missing, and unclear if both apply	“How do you usually travel to school? Walk, cycle, bus, car, other (please state)”
“What is your age?” with a blank line	Sensitive for some, hard to tally	“Which age group are you in? Under 18, 18–29, 30–44, 45 or over”

When the command word is “rewrite”, write the full new question and its options, not a description of the change.

Writing an advantage or disadvantage

Make each point a complete reason and tie it to the survey in the question.

• Weak: “Closed questions are quicker.”
• Better: “Closed questions are quicker to answer, so more of the 300 pupils are likely to finish the form, and the answers can be tallied straight away.”
• Weak: “Open questions give more.”
• Better: “An open question lets a member explain why they stopped using the pool, which a tick box cannot show.”

Pilot survey in steps

1. Write the first version of the questionnaire.
2. Try it on a small group like the real respondents.
3. Note blank answers, confused questions, odd answers, and the time taken.
4. Change wording, options, order or length.
5. Use the improved form in the main survey.

Purpose in one sentence: to find and fix faults before the full survey is carried out.

Response rate: worked reminder

150 forms are given out and 90 usable ones come back.

Response rate = 90 / 150 × 100 = 60%

To raise it: shorten the form, send a reminder, make it easy to return, explain why it matters, or offer a prepaid reply envelope. A bigger sample does not cure non-response bias or biased questions.

Must-know distinctions

• Closed vs open: closed has fixed options, open has none. Do not call a question open just because it has a long stem.
• Exhaustive vs mutually exclusive: exhaustive means nothing is missing, mutually exclusive means nothing overlaps.
• Pilot survey vs main survey: the pilot tests the questionnaire, not the population.
• Leading vs vague: leading pushes towards an answer, vague allows many meanings.
• Census vs sample survey: all or part of the population.

Quick self-test

1. State one advantage of an open question over a closed question.
2. State one disadvantage of a closed question.
3. A question has options “Under 20”, “20–30”, “30–40”, “Over 40”. Give one fault.
4. Why must open answers be coded?
5. Name the fault in “Is the cafeteria cheap and friendly?”
6. Name the fault in “Surely you would agree that homework should be banned?”
7. Give one purpose of a pilot survey.
8. A survey gives out 200 forms and 130 usable ones are returned. Find the response rate.
9. Of 40 coded answers, 14 mention “price”. Find the pie chart angle for price.
10. Rewrite “Do you read a lot?” as a closed question with a fixed time period.
11. Give two ways to raise a response rate.
12. Why does a larger sample not remove the bias from a leading question?

Answers

1. It lets the respondent give reasons or an answer the researcher had not thought of.
2. The respondent may find no option that fits, or the options may leave out an important answer.
3. 20 and 30 each appear in two options, so the options are not mutually exclusive.
4. The answers are in many different words, so they must be sorted into categories before they can be tallied and displayed.
5. Double-barrelled: it asks about price and friendliness at once.
6. Leading: the wording suggests the answer the writer wants.
7. To test the questionnaire on a few people so that unclear questions, missing options and length problems are found and fixed before the main survey.
8. 130/200 × 100 = 65%.
9. 14/40 × 360° = 126°.
10. For example: “In the last 7 days, how many books did you read? None, 1, 2 or 3, 4 or more.”
11. Any two of: shorten the form, send a reminder, give a prepaid return envelope, explain the purpose clearly.
12. Every extra answer is pushed in the same direction by the wording, so the error is repeated across the whole sample rather than averaged away.

Where marks are usually lost

• Naming a fault but not rewriting the question when the command word is “suggest” or “rewrite”.
• Writing “it is easier” without saying for whom or why (quicker to answer, quicker to tally).
• Giving the same point twice in different words when the question asks for two separate points.
• Writing bands that overlap or leave gaps in the corrected version.
• Saying a pilot survey “checks the results”. It checks the questionnaire.
• Giving a response rate using the number returned instead of the number usable.
• Leaving a loaded word in a “neutral” rewrite.
• Not showing the working for percentages and angles, and not stating the unit (° or %).
• Describing advantages and disadvantages of both types in general terms without applying them to the survey in the question.

Official syllabus

Cambridge O Level Statistics 4040, syllabus for examination in 2025, 2026 and 2027, published by Cambridge University Press & Assessment. Section 1.4: general ideas of surveys, including the use of closed and open questions in questionnaires.