There’s a simple technique to figuring out stuff that people usually get incredibly wrong. Here’s how it works.

Let’s say I give you a complex question with a bunch of moving parts and unknown factors. For example, “Is it feasible to create an original pokemon-style game with an additional mmo-like quest system, using a team of 7 exceptional students working part-time in under 8 months?”

The natural impulse is to start trying to solve the problem this way: “Okay, let me estimate how many features I can imagine the game requiring and how many individual art assets it would need. Then I’ll try to estimate how long each thing will take to produce, allow for bugs and glitches and figure out how much time it will take each student to fulfill these task lists.”

This is an exercise in futility. It’s the type of thinking that has spawned Hofstadter’s Law, which states, “It always takes longer than you expect, even when you take into account Hofstadter’s Law.”

Here’s how you do it right. It’s called Base-Rate Analysis.

You check how long similar projects took to complete. This is your base rate. After that, you check for significant external factors that would adjust the rate up or down. In this example your programmers might assure you that they can build the MMO-like quest system over the weekend, but no matter what it’s still an additional feature. Additional features take longer. Furthermore, your students may be exceptional but if you’re comparing them to a development schedule accomplished by a full-time professional team you have to expect it’ll take this team a lot longer.

Here are some times I’ve used this technique.

1. I predicted the iPad would be a success pre-launch. This might seem obvious now, but it was a very unpopular opinion at my business college. I figured it out by researching intended uses for it and found a hospital that was planning to buy them as a convenient way to cart around X-Rays and medical charts. That was a solution clearly worth the cost of the device, which meant there had to be more apps than this that would show up for other uses. The technology was only going to improve and the software was only going to expand. This formed my base-rate and I adjusted upward because of the potential.
2. I predicted No Man’s Sky would be a failure and said so the moment I read an article about it and the team. Why? Because it’s hard enough for any team to create a high-quality open world game with endless hours of enjoyable content. Doing it with a smaller team and bigger ambitions to create a whole universe was clearly adjusting the base rate to this being less and less feasible. Procedural generation for design adjusts down for quality as well, because for the moment humans are still better at designing than algorithms (and definitely better than a random indie team’s algorithms with no major experience in similar work).
3. While there are many videos debunking the Solar Roadways pitch using intricate equations and impressive research, you don’t need to go that deep. If you’re not familiar, Solar Roadways is a project that wants to make roads out of solar panels. The major barrier to adoption for solar energy is their cost-efficiency for the energy they produce. Solar Panels are already having trouble efficiently paying back their costs in ideal conditions on roof-tops and fields that track the sun. Now adjust that base rate by asking yourself how the added demands of being a durable road surface, with high traction in rain-slicked conditions, will affect their cost. Then ask how being a road surface getting covered in dirt and shadowed by hills and cars will affect their absorption of sunlight. Clearly this wasn’t going to work. So far it’s been a money pit, and a harmful one as the high-profile failure is draining resources and trust from workable projects.

This might sound like I’m boasting, but it’s the opposite. I’m demonstrating how simple it is to reach relatively complex and counter-intuitive conclusions once you apply this mental framework. It only produces impressive results because our instinctive mental processes kind of suck. This technique has several advantages, but one of its most significant is how neatly it sidesteps types of thinking humans are bad at. It doesn’t ask you to predict setbacks or roadblocks. In many cases it doesn’t even ask you for any subjective evaluation (humans suck at subjectivity in particular). It simply asks, “What’s something similar?” and then, “Based on objective factors, is this better or worse than that?”

I’ve used this technique innumerable times in my game dev career, particularly involving balance and project management. I’ve made it a habit when evaluating anything’s balance risk or playability to start by comparing it to some well-known existing base rate and then adjusting from there based on the most important differences between the base rate and the specific scenario under consideration.

Naturally I’ve gotten things wrong when using this technique. You still need to account for the correct factors. That’s still a lot easier than trying to imagineer every component of a project or build every possible deck when evaluating the balance of a new card.