Testing an Algorithm for Bias

Students trace a simple decision-making algorithm with varied test cases, evaluate whether its outputs are fair, and propose a revision that reduces bias.

Define Algorithmic Bias

An algorithm is a step-by-step process for turning inputs into outputs. Algorithmic bias occurs when that process creates a repeated, unfair advantage or disadvantage. Bias can come from the data, the selected inputs, the order of rules, or assumptions made by designers. Imagine an algorithm that selects students for a summer coding internship. It considers an assessment score, completion of a previous coding course, and a teacher recommendation. Previous courses may seem relevant, but not every school offers them. Teacher recommendations can also reflect different expectations or opportunities. If the algorithm consistently favors students who had greater access to courses or supportive adults, its outputs may be biased. A biased result does not require harmful intent. Testing is necessary because a rule can appear neutral while affecting groups differently.

Trace the Decision Rules

To test an algorithm, first trace its rules in the exact order in which they are applied. Consider this internship algorithm: If the assessment score is 80 or higher, accept the student. Otherwise, if the student completed a previous coding course and has a teacher recommendation, accept the student. In all other cases, place the student on the waitlist. Ava scored 76, completed a coding course, and received a recommendation. The first rule does not accept her because 76 is below 80. The second rule does accept her because both required conditions are true. Tracing prevents guesses about how a decision was made. It also reveals which inputs control the result. Record each comparison, branch, and output so another person can repeat the trace and verify the reasoning.

Run Varied Test Cases

A useful test set includes cases near important thresholds and cases that differ in only one input. Test the algorithm with five students. Maya scores 84 with no course and no recommendation, so she is accepted. Eli scores 76 with both a course and a recommendation, so he is accepted. Jordan scores 76 with no course but a recommendation, so Jordan is waitlisted. Sam scores 76 with a course but no recommendation, so Sam is waitlisted. Nia scores 79 with neither input, so she is waitlisted even though she is only one point below the cutoff. Eli and Jordan form a controlled comparison because their assessment scores and recommendations match while course completion differs. These varied cases help identify sensitive rules, edge cases, and patterns that one successful example would not reveal.

Compare Outcomes for Fairness

Fairness should be evaluated with stated criteria rather than personal preference. Useful criteria include consistency, relevance, equal opportunity, accuracy, and accessibility. The algorithm treats identical inputs consistently, but that alone does not make it fair. Eli and Jordan both scored 76 and had recommendations, yet only Eli was accepted because Eli had access to a previous coding course. If some schools do not offer that course, the rule may reward opportunity rather than ability. Teacher recommendations may also be affected by personal perspectives or unequal contact with teachers. However, different outcomes are not automatically evidence of bias; a difference can be justified when it is relevant to the internship's goals. Students should connect each fairness claim to test results and information about access. They should also identify missing evidence, such as course availability and later internship performance.

Revise the Algorithm

A revision should reduce the identified bias while preserving important goals. Remove previous course completion as an acceptance condition because course access is unequal. Instead, give every applicant the same short programming task using provided equipment and time. Score the assessment and work sample with published rubrics, then average the two scores. Accept students with an average of 75 or higher, and use a random lottery for tied applicants if space is limited. Two trained reviewers can score work samples without seeing names, and applicants should receive needed accessibility accommodations. For example, Jordan's assessment score of 76 and work-sample score of 82 produce an average of 79, so Jordan is accepted. The revision measures demonstrated skills more directly, although it requires more staff time and equipment. Continued testing is necessary because no single revision guarantees fairness.

Share Evidence and Reflect

Present the recommendation as a claim supported by evidence and reasoning. For example: The internship should replace the previous-course rule because controlled test cases showed that course access changed the output for applicants with the same score and recommendation. Explain how the revision addresses that problem, and acknowledge trade-offs such as added cost, scoring time, and possible reviewer disagreement. Also describe limitations: five test cases cannot prove that the revised system is fair for every applicant. Reflection should include different perspectives. Applicants may value equal access, clear reasons, accommodations, privacy, and a way to appeal. Program leaders may value accurate selection, efficiency, and limited costs. Public decision systems should respect human dignity and apply rules transparently. After implementation, leaders should compare outcomes, check scoring consistency, invite feedback, and revise the algorithm again if evidence shows unfair patterns.