Auditing a Decision Algorithm for Fairness
Students test a simple rule-based decision algorithm, compare its outcomes across groups, and revise its rules to make the results more accurate and equitable.
Illustrations are auto-generated and may be placeholders. They can be refreshed to match the narration.
How Decision Algorithms Work
A decision algorithm is a step-by-step set of rules that turns inputs into an output. Imagine a school algorithm that decides who may borrow a laptop for the weekend. It adds 2 points if a student has no home computer, 2 points for a major assignment, and 1 point for returning past devices on time. It subtracts 2 points if an adult cannot pick up the laptop after school. A score of 4 or more produces “Approve”; a lower score produces “Deny.” This algorithm always applies the same rules, but consistency alone does not guarantee fairness. A rule may be unrelated to the decision’s purpose or may create barriers for certain groups. Auditing means testing the algorithm, studying its results, and deciding whether its rules accurately identify students who need and can responsibly use a laptop.
Run the Algorithm on Test Cases
To audit the algorithm, create test cases with known information and record every calculation. Consider eight students: four bus riders and four students with other transportation. For bus rider A, no home computer adds 2, a major assignment adds 2, and timely returns add 1. Having no adult pickup subtracts 2, so A scores 3 and is denied. Bus rider B has the same needs but has adult pickup, so B scores 5 and is approved. Another bus rider, C, has no computer and a major assignment but no timely-return point and no adult pickup. C scores 2 and is denied. Testers should calculate all eight cases in the same way. They should also record a benchmark decision based on actual need and responsible use so they can identify correct decisions and errors.
Compare Outcomes Across Groups
Next, separate the results by group and compare both typical scores and decision rates. The bus riders have scores of 3, 5, 2, and 1. Their median score is 2.5, their range is 4, and only 1 of 4 students is approved, an approval rate of 25 percent. The other transportation group has scores of 5, 4, 3, and 4. Its median is 4, its range is 2, and 3 of 4 students are approved, an approval rate of 75 percent. A difference does not automatically prove unfairness, because groups may have relevant differences. However, the benchmark shows that three students in each group actually qualify based on need and responsible use. The algorithm correctly approves all three qualified students in one group but only one qualified bus rider. That pattern deserves investigation.
Identify Bias in the Rules
Bias can appear when a rule gives an unnecessary advantage or disadvantage. In this example, the adult-pickup rule lowers a student’s score by 2 points. Bus riders may be less likely to have an adult available at the required time because their families rely on school transportation or have fixed work schedules. Adult pickup does not measure laptop need or whether the student will use the device responsibly. The rule causes false negatives: students A and C meet the benchmark but are denied. To test whether this rule is the problem, compare similar cases. Students A and E both lack a home computer, have a major assignment, and returned devices on time. A lacks adult pickup and scores 3, while E has pickup and scores 5. This controlled comparison shows exactly how the pickup rule changes the outcome.
Revise and Retest the Algorithm
A fair revision should address the problem without ignoring safety or responsibility. Remove the 2-point penalty for lacking adult pickup and create a secure laptop handoff for students boarding school buses. Keep the need, assignment, timely-return, and 4-point threshold rules. Under the revised algorithm, the bus riders score 5, 5, 4, and 3, so three are approved. The other group still scores 5, 4, 3, and 4, so three are approved. These results match the benchmark: all six qualified students are approved, and the two students who do not qualify are denied. The revision improves accuracy and makes access more equitable. Retesting is essential because a proposed solution may create new errors. Auditors should use additional cases, check approval and error rates, and document why each rule supports the algorithm’s purpose.
