Pong Lab
The blue paddle learns with Q-learning, a classic reinforcement-learning algorithm. It is steered by a table of numbers, the Q-table, that starts empty. After every move the agent gets a reward: +1 when it wins a point, −1 when it loses one, 0 otherwise. It uses that reward to adjust one number in the table. Repeat that tens of thousands of times and it learns to play.
Each court is the same agent saved at a different point in its training, playing the same serve at the same time. Change a generation number (0 to 100) to compare any point in training, and click a court to watch it play, large, with its loop.
Training is 100 matches against the same opponent. Early on the agent explores, trying random moves 70 % of the time, and explores a little less after every match.
↑ Up ↓ Down • Stay
The agent never sees the screen. It sees a summary: where the ball will cross its line (wall bounces included), how far that is from the paddle's centre, and how soon it gets there. Similar moments share one row of the table, so there are only 65 rows.
+1 for a point won, −1 for a point lost, 0 for everything else. Nobody tells the agent to follow the ball. It discovers that moves which keep the ball in play lead to fewer −1s.
Each number in the table estimates the reward still to come after a move. The update pulls it towards what actually happened next: the reward plus the best value of the following state. Good news travels backwards, one move at a time.
An agent that only picks its current best move never finds a better one. During training it sometimes moves at random. The saved generations you watch never explore; they always take their best move.