Tetris ML
Tetris engine, DQN agent and training viewer · 2026A Tetris engine in Go, DQN agents trained with PyTorch, and a viewer that shows them playing and learning. Built to learn reinforcement learning by doing every layer myself, including an agent that uses hold, tucks, spins and T-spins.
- vs. heuristic
- +44%
- Avg score
- 173k
- T-spins / game
- 233
Problem: Reinforcement learning is easy to follow in a tutorial and hard to trust in practice. I wanted to learn it by building the whole pipeline: a game with rules exact enough to compare agents on identical games, a baseline worth beating, and a way to see what the agent actually learned.
Solution: A deterministic, seedable Tetris engine in Go (7-bag, SRS rotation and kicks, no rendering code) exposed over a gRPC stream, one game per stream. A Python Gymnasium environment sits on top of it and trains a DQN in PyTorch. A separate Ebitengine viewer plays human games, agent replays and training runs, and runs the exported networks itself without Python.
Technical Approach: Actions are placements (a rotation and a column per piece), and the network scores the board after a placement, the afterstate, from four features: aggregate height, holes, bumpiness and lines cleared. That shrinks the problem enough to train on a laptop. The Go and Python sides were verified against each other with a deterministic agent over 200 seeds, since a random agent never clears a line and so proves nothing. The viewer re-reads snapshots written every 5,000 steps, so it can follow a run live.
Results & Learnings: On held-out seeds with a 3,000-piece cap, the agent averages about 173,000 points against 119,880 for the hand-weighted heuristic. Two things did not help: a potential-based shaped reward collapsed mid-training before recovering, and Double DQN was within seed noise. The real limit was measurement: a cap hides differences between strong agents, so I added score per piece and a 10,000-piece survival check. A second agent searches every reachable lock position and learns T-spins, scoring about 240,000 points with 233 T-spins per game, but only when its network cannot see the board, and its checkpoints vary more within one run than the gap to a scripted baseline. The full write-up covers what failed as well as what worked.
- Stack
- Go · Python · PyTorch · Gymnasium · gRPC · Ebitengine
- Role
- Full implementation