What this is
A complete 1v1 fighting game (deterministic 60 Hz simulation, procedural stickman animation, online play) wrapped around a reinforcement-learning pipeline. The same JavaScript environment module runs in three places: the browser (playing), the server (online relay), and behind a stdio bridge (Python training). One implementation, zero simulation drift between training and inference.
Architecture
┌────────────────────────────┐
│ game/env.js (pure JS) │ deterministic sim, 60 Hz, no rendering
└─────────────┬──────────────┘
┌─────────────────┼──────────────────────┐
▼ ▼ ▼
browser client node relay (rooms) node bridge (stdio)
render + input + lobby + input relay "env as a service"
vs-AI / spectate (rollback netcode) for Python training
▲ ▲
│ weights/*.json (MLP) │ JSON over stdin/stdout
└──────────── training/ (PyTorch) ───────┘
The brain is exported as plain JSON and loaded by a ~100-line from-scratch forward pass in the browser — no ML library runs client-side. A parity test replays the exported brain through the browser inference code and requires it to keep beating the bot, proving the train→export→inference chain.
How the AI learns
Observation — 24 dims
| HP (self / opp) | 2 |
| spacing dx, |dx|, dy | 3 |
| velocities vx, vx_opp, vy | 3 |
| self state one-hot (7 states) | 7 |
| opponent state one-hot | 7 |
| attack frames remaining ×2 | 2 |
Actions — 7 discrete
idle · move left · move right · jump · punch · kick · block
Reward
+0.1 × damage dealt
−0.15 × damage taken
−0.001 / frame (urgency)
±10 win / loss
Double-DQN
Adam 3e-4 · γ 0.995 · replay 200k · batch 128 · target sync 2k steps ·
ε 1.0→0.05 over 500k frames · smooth-L1 loss.
Self-play: both fighters share the current net; 25% of episodes use a frozen
snapshot opponent (pool of 5) to avoid strategy cycling.
Results
Every checkpoint exported during training (every 100k frames) plays 30 episodes against the scripted bot (difficulty 0.85, cap 30 s). Reference: a purely random policy wins 0% of its matches.
The win rate isn't monotone — the policy swings while it plays against itself. The first checkpoint (100k) already wins 57% of its matches with a defensive block-spam (36% of its inputs) and counter-punches. Self-play then pushes it into a turtling local optimum: at 500k it blocks 61% of the time and never attacks, so episodes stall to the cap and only 5% are wins. It escapes that trap from 600k on — the final brain is active and mobile (31% forward, 21% jump, 18% punch), winning 100% of its matches with KOs in ~8.5 s and 87% of the damage dealt. The final check (50 episodes each): 100% vs bot 0.5 · 100% vs bot 0.85 · 96% vs bot 1.0.
win rate vs scripted bot (random baseline = 0%)
average episode length — wins become fast KOs instead of slow grinds (cap = 30 s)
share of damage dealt vs received — learns to trade favourably
training loss (smooth-L1, EMA) over 1M frames
Learning progression
Greedy matches recorded from brains exported during training (same seed, vs the scripted bot). Early checkpoints mostly lose — the last one wins.
Netcode — GGPO-lite rollback
Online 1v1 uses rollback: both clients simulate the deterministic env locally at 60 Hz and exchange framed inputs through the server (a dumb relay). Local inputs apply instantly; the peer's inputs are predicted (repeat-last) and, when reality disagrees, the session rolls back a few frames and re-simulates — a handful of tiny steps that are invisible in practice. Rooms are gated by an access key with per-IP rate limiting.
Stack
Vanilla JS ES modules (no build step) + Canvas 2D · Node 20 + ws (only dependency) ·
PyTorch (CPU) + numpy · nginx + Let's Encrypt · systemd on an Oracle free-tier VPS ·
zero-dep tests via node:test (17 tests: determinism, combat rules, bridge
protocol, brain parity, netcode).