Morpha
A card game built to test one thing: how reinforcement learning copes when the set of legal actions changes every turn, and new actions can be created mid-game.
The problem
Most reinforcement learning assumes a fixed menu of actions. In Atari, chess or Go, every move can be given a permanent ID up front, and the agent picks a number from that list. When a move isn't allowed, it's masked out.
Many real problems don't work that way. In card games, inventory systems, tool use or program synthesis, the legal actions depend on the state. They carry meaning ("deal 3 fire damage"), and they can be combined into new actions that never existed in any list. Masking a giant fixed list hides that structure from the agent and stops scaling once the combinations explode.
The game
Each turn you get energy and a hand of cards. Cards deal kinetic, fire, poison, ice or thunder damage, heal, draw, or change the rules for later turns. The key mechanic is the morph: any two cards in hand can be fused into a new card that combines both effects. The number of possible actions therefore grows combinatorially with the hand, and many of them are new to the agent.
The game spans three eras of cards (alchemy, a research lab, a space station), each adding new effects. The browser version runs the same Python engine the agents train on, compiled to WebAssembly with Pyodide. Its PvE mode is a damage puzzle: break the training dummy before the turns run out.
You can also duel the trained agents, or watch one play and take over mid-game. George Jr is a PPO agent and Quebert a DQN agent. Their policies are exported to ONNX and run in your browser with ONNX Runtime. Each is about 7 MB and downloads the first time that agent moves. Azasel, the AlphaZero agent, and the agents that read the cards' text still need the locally served version.
Five independent difficulty knobs
The benchmark separates how hard the game is from how hard the action space is, so each can be scaled on its own and the failure points of an algorithm can be located.
| Axis | What it scales |
|---|---|
| A Action space | Legal actions per state, branching factor, how many combinations morphing creates |
| H Horizon | How far ahead plays pay off: delayed effects, health targets, round caps |
| S Semantics | How many distinct effects and interactions the cards carry |
| O Observability | Hidden information and random events |
| G Game theory | From a passive target to a full adversarial opponent |
What's being compared
The same games are played by agents that see a static, masked action list and by agents that read dynamic, described actions: PPO, DQN and AlphaZero in both forms. The hypothesis is that methods which use the meaning of the actions scale much better than those relying on fixed IDs as the action space grows. EidolonZero is my model-based planner built for this setting.
Results will appear here as the experiments finish.