Tetris ML

Tetris engine, DQN agent and training viewer · 2026
Move agent, 100,000 training steps[dqn]

A Tetris engine in Go, DQN agents trained with PyTorch, and a viewer that shows them playing and learning. Built to learn reinforcement learning by doing every layer myself, including an agent that uses hold, tucks, spins and T-spins.

vs. heuristic
+44%
Avg score
173k
T-spins / game
233

Problem: Reinforcement learning is easy to follow in a tutorial and hard to trust in practice. I wanted to learn it by building the whole pipeline: a game with rules exact enough to compare agents on identical games, a baseline worth beating, and a way to see what the agent actually learned.

Solution: A deterministic, seedable Tetris engine in Go (7-bag, SRS rotation and kicks, no rendering code) exposed over a gRPC stream, one game per stream. A Python Gymnasium environment sits on top of it and trains a DQN in PyTorch. A separate Ebitengine viewer plays human games, agent replays and training runs, and runs the exported networks itself without Python.

Technical Approach: Actions are placements (a rotation and a column per piece), and the network scores the board after a placement, the afterstate, from four features: aggregate height, holes, bumpiness and lines cleared. That shrinks the problem enough to train on a laptop. The Go and Python sides were verified against each other with a deterministic agent over 200 seeds, since a random agent never clears a line and so proves nothing. The viewer re-reads snapshots written every 5,000 steps, so it can follow a run live.

Results & Learnings: On held-out seeds with a 3,000-piece cap, the agent averages about 173,000 points against 119,880 for the hand-weighted heuristic. Two things did not help: a potential-based shaped reward collapsed mid-training before recovering, and Double DQN was within seed noise. The real limit was measurement: a cap hides differences between strong agents, so I added score per piece and a 10,000-piece survival check. A second agent searches every reachable lock position and learns T-spins, scoring about 240,000 points with 233 T-spins per game, but only when its network cannot see the board, and its checkpoints vary more within one run than the gap to a scripted baseline. The full write-up covers what failed as well as what worked.

Stack
Go · Python · PyTorch · Gymnasium · gRPC · Ebitengine
Role
Full implementation