AI / LLMs are terrible at designing game systems. Here’s an example:

I asked Gemini Pro to design a progression system for “Chess – The RPG”, bearing in mind it needs to work for PvP too, keeping a summary to 2 paragraphs. It gave me this.

Progression: In “Chess – The RPG,” advancement centers on individual piece development and a meta-skill tree to create unique, asymmetrical builds. Players start with a basic set of pieces that earn XP by surviving matches, capturing enemies, and executing checkmates. Upon leveling up, pieces unlock divergent class promotions; for example, a standard Pawn could branch into a “Shieldbearer” that absorbs one capture attempt, or a “Rogue” with an extended diagonal backstab mechanic.

Winning matches also grants “Grandmaster Points” used to unlock overarching player abilities on a central skill tree, granting limited-use tactical spells like “Forced March” (extra movement) or “Resurrection,” ensuring no two armies bring the exact same loadout to the board.

Each of those Mechanics is a cool idea, but they fail as a System. They undermine and frustrate eachother, and would waste months of developer time. Some of the many issues:

1. Boards will become un-parseable with every piece levelling up independently, while keeping overall customization/identity strangely low for such a big complexity cost.

2. Instead of unlocking options as a form of progression, players are being forced to upgrade pieces – which creates more guarunteed imbalances on ladder.

3. Weird incentives, like trying to keep pieces alive unti to gain more XP for them. Trading is a counter-balance, but better to just incentivize the good gameplay. Resigning on turn 1 might actually be the best way to get the “survival” bonus on all your pieces.

4. Skill tree progression comes out of nowhere, and multiplies complexity. No one will be able to track all their opponent’s options, and the unique sets of options will mean you can’t get familiar with them over time.

5+. There’s more. See if you can spot them yourself.

Better Solve: If it was me, I’d be looking for ways to add an RPG-like sense of class or character progression, with lots of potential depth, without making PvP an unparseable/unbalanceable mess of branching board setups. I’ve only thought about this problem for 5 minutes, but it seems fair to compare my top-of-head design to gemini pro’s few minutes of thinking too.

Top of my head: Make “Generals” or “Classes” that have dedicated armies with unique combinations of pieces. Level the general by playing games, with XP based linearly on the total number of moves played with a small bonus for time spent and then a multiplier if you win. This would reward people primarily for spending time actively playing the game, prevent instant resignations for XP farming, and still create an incentive to win.

As you level up, you unlock alt-pieces you can use to lightly customize this general’s army and upgrades to existing units – giving each general some unique pieces or combinations. Perhaps one general has four Knights, while another has a more powerful version of the king that is worth bringing into battle early instead of protecting the whole game. Some generals with weaker pieces get a passive or a top-bar.

This solution adds RPG-identity + progression to chess in a clear, non-disruptive way. It’s better for PvP balance, matchup stability, and progression depth too. PvP players would be able to learn how to deal with the different generals/classes and their standard setups over time, rather than trying to parse the endless granularity of specific pieces having their own divergent upgrade paths and progression states combined with the grandmaster-point-skill-tree of top-bar spells that was Gemini Pro’s proposal. It’s also much easier to balance “bundles” of options vs trying to balance all those individual options separately.

Leaning on the identity idea, this also puts emphases on the RP part of RPG: Building and progressing characters. While you could turn each piece into a unique character, this gets out of hand fast in the calculation-heavy matchups that make up a chess game. Players often want to pour themselves into a “main character” anyway, and by creating lots of different ways to express that character’s unique identity through its class and upgrade choices, this scratches at the implied design goals of “Chess the RPG” far better than making the same “Shieldbearer or Rogue” divergent path upgrade choice for 8 different pawns (even if that wasn’t a complexity nightmare).

This approach is also stronger in an additional way: far more progression depth. Instead of 16 pieces to upgrade and one skill tree, each general/class can have its own upgrades. There’s much more total progression depth for completionists here, and if the game wanted to incentivize completionism across the board they could create special challenges that incentivized using multiple different generals or shifted which are better in the current special event. The opportunity is there either way, and opportunities are nice to have.

Again, this is just top of the head with a few minutes of thinking. Saving months of bad system design implementation is worth weeks of a single system designer’s time, but I really didn’t need more than a few minutes to come up with something much better.

AI can’t be relied on for system design. You will occassionally get a workable answer when it copies a summary of systems from some highly specific game, but it will always miss the meaningful context and it will fail whenever it encounters a novel problem that demands some contextual intelligence. It will not be able to see how its basket of disconnected ideas interact.