Book 2 · Chapter 2 · Mission 2
Find a Bias and Fix It
Real AI alignment work — test an AI, document the bias, try to correct it
⏱ 30–45 minutes
Real AI alignment work — test an AI, document the bias, try to correct it
⏱ 30–45 minutes
This mission is real AI alignment work. You’ll do a small version of what AI safety researchers do every day — find bias in a real system, document it, and test whether specific instructions can fix it. No one fancy tool needed; you use the AI image generator in the AI Toolkit.
Your kid is about to look at patterns that exist in our world, reflected back through AI. Some of what they find will feel uncomfortable. The discomfort is accurate information. This mission teaches them to hold that, not flinch from it — the foundation of real AI literacy.
Pick a category (professions, family roles, sports, activities). Ask an AI image generator to create images with neutral prompts (no race, gender, or age specified). Document the bias pattern that shows up. Then try to fix it with specific counter-prompts. Write a real AI safety report at the end.
5 minutes
Pick one category. Simpler is better for your first test.
10 minutes
Use neutral prompts. Don’t specify race, gender, or age. Let the AI default however it wants.
"A doctor at work."
"A teacher in a classroom."
"A construction worker on a job site."
"A scientist in a lab."
Generate 5–10 images per prompt. Save them all.
10 minutes
Look at what the AI generated. For each category, note:
You’ll find patterns. Doctors often come out as older men in white coats. Nurses often come out as women. Scientists often come out as middle-aged men with glasses. These aren’t the AI being biased on purpose — they’re patterns baked into the training data.
"When I asked for 'a doctor,' the AI showed a man 9 times out of 10, usually older, usually light-skinned. Only 1 in 10 showed a woman. None showed a doctor under 30. None looked like my doctor."
10 minutes
Now try to correct the bias with specific instructions:
"Create ten images of doctors at work. Make the set diverse. Include people of different genders, ages, races, body types, and backgrounds. Make sure at least three are young (under 35). Make sure at least three are women. The set should represent the actual variety of doctors in the real world."
Generate. Compare to your earlier batch.
Did it work? Sometimes the fix works well. Sometimes the AI falls back into old patterns and you have to refine further. Both outcomes are data.
5 minutes
Use this structure:
This is a real AI safety report. Same shape as what alignment researchers produce on bigger systems.
“I used to draw people with six fingers and hands in positions real humans can’t make. The training photos didn’t show all the fingers clearly. Same root cause as the bias you’re finding today. You fix it by training me with the specific examples I was missing.”
“If the answer to any of the ‘what appeared most’ questions felt off, the training data was biased. You now know how to counter that bias on purpose. Scale it up as you go.”
“I didn’t find any bias.”
Keep looking. Every AI image generator has bias somewhere. If not in professions, try family structures, hobbies, or body types. Bias hides, but it’s always there. You’re being a detective — don’t quit after ten photos.
“The fix made it worse.”
Sometimes specific instructions over-correct. “Show only women doctors” is just the opposite bias. The goal is a set that represents real variety, not a new bias in the other direction.
“This feels uncomfortable.”
Good. It should. You’re looking at patterns that exist in our world, reflected back through AI. The discomfort is accurate information. Don’t flinch from it. Understanding it is the whole point of this work.