Testing Algorithms for Bias

Students use test cases to identify unfair patterns in a simple decision-making algorithm and propose a more equitable set of rules.

How Algorithms Make Decisions

An algorithm is a step-by-step set of instructions for completing a task or making a recommendation. People design algorithms, choose their goals, and decide which information matters. Therefore, an algorithm can reflect human assumptions and perspectives. Imagine a school uses an algorithm to assign popular study rooms. The rule gives priority to students who submitted requests before 8:00 a.m. This may sound neutral, but students who ride early buses might not have internet access at that time. To evaluate the algorithm, ask what claim supports the rule and what evidence shows that the rule helps students fairly. A computer follows its instructions consistently, but consistency alone does not make a decision fair. Testing helps people discover who benefits, who is disadvantaged, and whether the rules support shared values such as equal opportunity.

Inputs, Rules, and Outputs

Most decision-making algorithms have three main parts: inputs, rules, and outputs. Inputs are the data given to the algorithm. Rules are the instructions used to process that data. Outputs are the resulting decisions, scores, or recommendations. Suppose an after-school robotics club algorithm uses attendance, completed projects, and access to a ride home as inputs. Its rule admits students with strong attendance and a guaranteed ride. The output is “admit” or “do not admit.” The transportation input may create unfairness because reliable transportation often depends on family circumstances rather than interest or effort. When examining an algorithm, trace each output back through the rule to the inputs. Ask whether every input is relevant to the goal and supported by evidence. Also consider whether an apparently neutral input could act as a substitute for personal or economic circumstances.

Testing with Varied Cases

A test case is a set of sample inputs used to check what an algorithm produces. Good testing includes varied cases, edge cases, and paired cases. In a paired test, two cases are identical except for one input, making that input’s effect easier to see. For example, test the robotics club algorithm with two students who each attended 90 percent of school days and completed three projects. One has a guaranteed ride, while the other uses the late bus. If only the first student is admitted, the test reveals that transportation changes the result. One example does not prove a broad pattern, so testers should run many cases and record results in a table. They should use the same criteria each time, such as equal treatment of similarly qualified students, connection to the club’s purpose, and the practical limit on available spaces.

Recognizing Unfair Patterns

An unfair pattern appears when an algorithm repeatedly disadvantages certain people for reasons that are not relevant to its stated goal. Testers should look beyond one surprising result and compare outcomes across many cases. Suppose 40 students qualify for the robotics club based on attendance and completed projects. The algorithm admits 18 of 20 students with private transportation but only 4 of 20 students who depend on the late bus. This difference is evidence of a possible unfair pattern because transportation does not measure robotics ability or commitment. However, the numbers alone do not explain the cause. Testers must examine the rules, check data quality, and consider other explanations before making a claim. They should also listen to affected students, whose experiences may reveal barriers the designers overlooked. Fair evaluation combines numerical evidence, clear reasoning, and perspectives from the school community.

Redesigning and Comparing Rules

After finding a possible bias, designers can propose new rules and compare them systematically. Useful criteria include fairness, accuracy, relevance to the goal, transparency, and ease of use. Constraints might include a 20-student club limit, staff time, or available transportation. For the robotics club, one redesign could remove the guaranteed-ride requirement and arrange access to the late bus. Qualified students could then be ranked using relevant evidence, such as completed projects and demonstrated interest. If equally qualified students exceed the space limit, a transparent lottery could choose among them. Test both the original and redesigned algorithms with the same varied cases. Record how each performs on every criterion rather than declaring a winner from one result. The redesign should also be discussed with students, families, and staff. Including different perspectives supports democratic decision-making and helps the school choose rules that are practical, explainable, and more equitable.