Set aside the test cards before you start

In machine learning, a system can learn patterns from examples with known labels. Those examples make up the training data. A separate test set checks how the system does with new examples. In the club activity, one person plays the role of a classifier that follows a rule. Ask another member of the unit to hold back the test cards until the rule is ready. Code.org, training data and bias · Google for Developers, dividing training and test data

Advertisementpreview

Make six training cards. Draw three different torches or lamps on blue backgrounds and three different water bottles or canteens on yellow backgrounds. Write the correct category on the back of each card: “LIGHTING” or “WATER.” Prepare four new drawings for the test and keep them out of sight. Use a headlamp on yellow, a canteen on blue, a handheld torch on blue and a water bottle on yellow. The colour to watch is the card’s background.

Find out what happens when the classifier uses colour

Show the group the six training cards and their labels. The person playing the classifier uses this rule: “Blue background means lighting. Yellow background means water.” It gets all six training cards right. Now bring out the four unseen cards one at a time. Ask for a prediction before flipping each card over to check. Have another person record the result.

Expect two correct answers out of four. The headlamp on yellow is assigned to water. The canteen on blue is assigned to lighting. Look back at the training cards and ask whether any of them gave the class a yellow-background example of a light. There were none. The group can now explain how a rule that fit every training card still made mistakes. Google for Developers, why test on unseen examples

Read alsoCambodia’s krama: a paper weaving workshop for Pathfinders
Advertisementpreview

Try new training examples and a more relevant clue

Make new training cards with both types of equipment on both backgrounds. Ask the team to propose a clue that matters for sorting equipment. For example, does the object produce light or hold water? Draw four more unseen objects for a fresh test after settling on the rule. Compare the decisions and keep a note of any object your rule struggles to assign.

Decide what to do when an object fits both categories

A water bottle with a built-in light might fit both categories. Try a “check with the unit” label for a card like that. Ask each pair to explain which details helped its rules and which new examples it would draw next. The paper model lets you see what the choice of examples can change. Real systems need a larger and more varied test set to assess how well their predictions work. Code.org, how training examples affect predictions

Further reading: Pathfinder Wiki, Artificial Intelligence honor requirements