Reinforcement learning
GridWorld AI
A tabular Q-learning agent learns the optimal path through a 6x6 maze - dodging a hazard - purely by trial and error. It starts knowing nothing; there is no hand-coded pathfinding, only reward and 8,000 episodes of practice.
How to read it: the agent gets +1.0 for reaching
the goal G, -1.0 for the hazard X,
and a small -0.04 per step so dawdling costs. The greedy path it settled on is drawn
with o.
The maze
Column x=2 is a near-solid wall, open at one gap, so every route
from S to G must funnel through it. The hazard X
sits right off the final approach to the goal.
What it learned
The greedy policy (epsilon=0) after training - o
marks every floor cell the agent walks through, dodging the hazard:
How Q-learning works
The agent keeps a table Q(state, action) estimating how good each move is.
After every step it nudges its estimate toward what it just observed - the textbook
Bellman update:
Q(s,a) <- Q(s,a) + alpha * [ r + gamma * max Q(s',a') - Q(s,a) ]
While training it explores with an epsilon-greedy policy (mostly random early, mostly exploiting later). Because the update always bootstraps off the best next action, Q-learning is off-policy - it converges on the optimal policy even from a partly-random data stream. This is Sutton and Barto's textbook algorithm (Reinforcement Learning: An Introduction, 2e, section 6.5).
Actual run output
Training 8000 episodes (alpha=0.2, gamma=0.95, epsilon 1->0.05)... Episode-return trend (16 buckets averaged over the 8000-episode run): episodes 0- 499 : avg return = -7.231 episodes 500- 999 : avg return = -5.767 episodes 1000- 1499 : avg return = -4.025 episodes 1500- 1999 : avg return = -2.813 episodes 2000- 2499 : avg return = -2.132 episodes 2500- 2999 : avg return = -1.472 episodes 3000- 3499 : avg return = -1.050 episodes 3500- 3999 : avg return = -0.833 episodes 4000- 4499 : avg return = -0.555 episodes 4500- 4999 : avg return = -0.376 episodes 5000- 5499 : avg return = -0.303 episodes 5500- 5999 : avg return = -0.114 episodes 6000- 6499 : avg return = 0.020 episodes 6500- 6999 : avg return = 0.109 episodes 7000- 7499 : avg return = 0.233 episodes 7500- 7999 : avg return = 0.325 Goal reached in 5984/8000 training episodes (74.8%); final epsilon=0.050 Greedy rollout: 16 steps, total reward=0.400, reached terminal=True SUCCESS: the trained agent reaches the goal G in 16 steps (optimal is 16), avoiding hazard X.
Deterministic (fixed PCG32 seed) - every run prints byte-identical output. A console/SSR render of that program's real output; BCL-only, warnings-as-errors, 0/0 build.