Using Programs with Data: Study Guide (Computer Science Principles 2.4)

Why use a program at all?

Topic 2.3 described what we can learn from data. This topic is about how. A program can process far more data than a person, and it can repeat the same operation exactly. The usual way of working is iterative and interactive: you ask a question, the program processes the data, you look at the result, and then you refine the question and run it again. For example, you filter customers by region, see a chart, notice an odd bar and filter again by month. Each pass gives more insight and knowledge about the data.

Programs let people discover information and create new knowledge, but they do not decide what the answer means. That is still a human judgement, and it depends on the quality of the data (Topic 2.3).

Four processes on data

The course describes some processes that extract or modify information from data. Each can be written as a short algorithm over a list.

Process	What it does	Example
Transform	Applies the same operation to every element	Double every number; add a parent’s email to every student record
Filter	Keeps only elements that meet a condition	Keep the scores that are at least 60
Combine or compare	Reduces many elements to one result, or compares them	Add up a list; find the highest score
Visualise	Shows the data as a chart, graph or other picture	A bar chart of totals per club

Other ways programs help gain insight are combining data sources (joining two tables on a shared identifier), clustering (grouping similar items) and classifying (assigning items to categories). Search tools find information efficiently, filtering systems pick out patterns, and spreadsheets organise data and make trends easy to see. Often a pattern emerges only after the data have been transformed or visualised.

A note on the pseudocode

The examples use Marlbridge’s rendering of the course’s reference notation: lists are written in square brackets, FOR EACH item IN list { ... } visits each element in order, APPEND(list, value) adds to the end, LENGTH(list) gives the number of elements, and MOD gives a remainder. Lists are covered fully in Unit 3; here you only need to trace them. Use the official reference sheet your teacher provides for the exam.

Tracing list-processing code

A filter builds a new list from the elements that pass a test:

scores ← [72, 45, 88, 91, 54]
passed ← []
FOR EACH s IN scores
{
  IF (s >= 60)
  {
    APPEND(passed, s)
  }
}
DISPLAY(passed)

Tracing means recording the value of the loop variable and the list after each pass. A transform looks similar but has no IF, so every element is used:

doubled ← []
FOR EACH s IN scores
{
  APPEND(doubled, s * 2)
}

A combine keeps a running total:

total ← 0
FOR EACH s IN scores
{
  total ← total + s
}
DISPLAY(total)

Worked example 1: tracing a filter and a combine

Task. Trace this code, give the output, and state what it does.

scores ← [72, 45, 88, 91, 54]
total ← 0
count ← 0
FOR EACH s IN scores
{
  IF (s >= 60)
  {
    total ← total + s
    count ← count + 1
  }
}
DISPLAY(total)
DISPLAY(count)

Step 1: trace.

s	s >= 60?	total after	count after
(start)		0	0
72	yes	72	1
45	no	72	1
88	yes	160	2
91	yes	251	3
54	no	251	3

Step 2: output. DISPLAY(total) shows 251 and DISPLAY(count) shows 3.

Step 3: what it does. It filters the scores to those at least 60 and combines them, adding them up and counting them. In words: “displays the sum of the passing scores and how many there are”.

Check. Verify by hand: the passing scores are 72, 88 and 91, whose sum is 72 + 88 + 91 = 251, and there are 3 of them. Notice that the two failing scores, 45 and 54, never change either variable. A teammate wanting the average of the passing scores could compute 251 ÷ 3 after the loop (about 83.7).

Worked example 2: an iterative analysis of two linked tables

Scenario. A fictional running league keeps two tables, combined using the runner number as a shared key.

Runner	Club
1	Hawks
2	Owls
3	Hawks
4	Owls
5	Hawks

Runner	Kilometres this week
1	12
2	7
3	20
4	15
5	9

Task. Find out which club ran more, and refine the question once you see the first result.

Pass 1: combine and add. Join the tables on the runner number: 1 Hawks 12; 2 Owls 7; 3 Hawks 20; 4 Owls 15; 5 Hawks 9. Total per club: Hawks 12 + 20 + 9 = 41 km; Owls 7 + 15 = 22 km. A bar chart (visualising) shows Hawks almost twice as tall.

Pass 2: refine. The chart prompts a question: do the Hawks run more only because they have more runners? Hawks have 3 runners and Owls have 2. Compute the average per runner: Hawks 41 ÷ 3 = about 13.7 km; Owls 22 ÷ 2 = 11 km. The gap is much smaller.

Pass 3: filter. Another question: how many runners in each club ran at least 10 km? Filter the combined data to runners with 10 km or more: runner 1 (Hawks), runner 3 (Hawks) and runner 4 (Owls). Count per club: Hawks 2, Owls 1.

Interpretation. The totals alone overstated the difference; after adjusting for club size, the clubs are closer. Because the dataset has only five runners for one week, we should be careful about concluding anything about the clubs in general. This is the iterative and interactive process: each result suggested the next step.

Three patterns that appear again and again

Most list-processing code is a variation on one of three patterns. Learn the shape of each, and you can read unfamiliar code quickly.

1. Count or sum. Start a variable at 0 before the loop and add to it inside the loop (adding 1 for a count, or adding the element for a sum). Forgetting to start at 0, or resetting it inside the loop, are the usual mistakes.
2. Filter. Put the action (for example APPEND) inside an IF inside the loop.
3. Find the largest (or smallest). Start with the first item, then replace it whenever something larger appears.

Here is the third pattern for a list values ← [14, 7, 22, 9]:

largest ← values[1]
FOR EACH v IN values
{
  IF (v > largest)
  {
    largest ← v
  }
}
DISPLAY(largest)

(In the reference notation, values[1] is the first item; list positions start at 1.) Trace: largest starts at 14. For v = 14, 14 > 14 is false. For v = 7, false. For v = 22, 22 > 14 is true, so largest becomes 22. For v = 9, 9 > 22 is false. The output is 22.

Two cautions. First, if the loop started largest at 0, the code would give wrong answers for a list of negative numbers, so starting from the first item is safer. Second, the pattern fails on an empty list, because there is no first item: that is a run-time error, which a test with an empty list would reveal. Thinking of the pattern and then its edge cases is the quickest way to check any such code.

When you answer a question about processing data with a program, say what each step does to the data, and not only its name. “Filter the loans to those from March” tells the reader which records remain, and “add up the loans per student” tells them what number is produced. Names alone are easy to guess; descriptions show understanding.

Common misconceptions

• “A program tells you what the data mean.” A program processes data; people interpret the result, using context and care.
• “Filtering changes every element.” Filtering keeps some elements; transforming changes every element.
• “One pass is enough.” Analysis is usually iterative; the first result often suggests a better question.
• “A chart proves a conclusion.” A visualisation reveals a pattern; the pattern may reflect small or biased data.
• “Combining data sources always makes data more accurate.” It can add insight, but formats must match, and errors in either source carry over.
• “Transforming removes elements.” A transform of every element keeps the same number of elements.
• “The order of steps never matters.” Cleaning before counting, or filtering before averaging, can change the answer.

Where this leads

This completes Big Idea 2. Unit 3 shows how to write the lists, loops and procedures that carry out these processes, and Unit 5 returns to the dangers of biased data and the privacy of personal data. Try the Unit 2 diagnostic when you have worked through the practice questions, the revision notes and the checklist.