Short on time? This page is the recap. For explanations and worked examples, use the full study guide.
Recap
- Computing innovations can reflect existing human biases through biased algorithms or biased data.
- Biases can be embedded at all levels of software development.
- Proxy: a feature closely linked to a characteristic that can have the same effect as using the characteristic.
- Detect bias by computing results for each group and comparing; an overall figure can hide a gap.
- Reduce bias: representative data, testing across groups, reviewing rules, varied perspectives, human review, monitoring after release.
- Bias is not eliminated by simply collecting more data from the same source.
Key relationships
| Idea | How to recognise or check it |
|---|---|
| Biased data | One group is missing or under-represented, or past decisions were unfair |
| Biased algorithm | A rule or formula favours some groups, perhaps through a proxy |
| Group accuracy | correct ÷ tests, for each group |
| Overall accuracy | total correct ÷ total tests (a weighted average) |
| Gap | difference between group results, in percentage points |
Assumptions and conventions
- Percentage point differences: subtract the percentages (96% - 60% = 36 percentage points).
- Say which group is affected and how, and keep the language neutral.
- A fix should be specific (what data, what test, what rule change).
Mistakes to avoid
- Reporting only the overall figure.
- Claiming a program is fair because the code does not mention any group.
- Suggesting “more data” without saying that it must be representative.
- Forgetting that bias can arise at any stage.
- Blaming only individuals when the data or design are the source.
- Proposing a fix that is not specific.
Two quick worked examples
1. A system is right on 45 of 50 tests for Group P (90%) and 20 of 40 for Group Q (50%). Overall it is right on 65 of 90 tests, about 72.2%. The overall figure hides a 40 percentage-point gap.
2. A hiring rule requires “at least 5 years in the same job”. Applicants with career breaks, for reasons such as caring for family, are screened out. The rule never mentions the group, but it may act as a proxy, so the company should check results by group and ask whether the rule is needed.
A revision routine
Write one example of biased data and one of a biased rule. Then invent test results for two groups of different sizes, calculate each group’s accuracy and the overall accuracy, and write the gap in percentage points. Finish by listing three actions that reduce bias and saying which kind of bias each addresses.
Quick self-check
- A system is right on 45 of 50 tests for Group P and 15 of 25 for Group Q. Accuracy for each? (90% and 60%.)
- Why might “postcode” be a poor input for a loan decision? (It can act as a proxy for characteristics that should not matter.)
- Name two ways to reduce bias. (For example, representative data and testing results by group.)
Next: practice questions.