We are the worst judges of our own difficulty score
Our genre model has five numbers, and four of them describe the market: how validated a mechanic is at home, how crowded it is where we would ship, how much it leans on language, how much regulatory trouble it drags along with it. One of them describes us. It is the only number in the model whose subject has a personal interest in the answer, so it is the one I have gone at hardest, and it is the one where I keep catching myself cheating.
Why D is different
D is what it costs one person to ship a playable version. Every other score in the model can be argued about with a link. D cannot. D is a claim about my own capacity, made by me, at the exact moment I most want the project to look attractive, which is the moment a person is least reliable about their own capacity. I have never once estimated a build at the number it turned out to be, and the misses are not symmetric, because every one of them has landed on the optimistic side, including the ones I was certain I had already corrected for.
The scale is 1 to 10 and the basis is deliberately about failure modes rather than effort, since “how hard does this feel” is not a question anybody answers honestly about their own work. Real time multiplayer or an enormous volume of content puts you at 8 to 10. Party and social competition games sit at 9 for both reasons at once, needing netcode and an endless supply of levels, and that genre scored 5.56, comfortably inside our top six, and we threw it out.
Survival strategy is a 9 for a less obvious reason: the genre runs on long term guild ecology, which is a community operations job wearing a game’s clothes, and no amount of clever engineering substitutes for somebody answering messages at midnight. Merge with story is a 7, not because merging is hard but because the format wants a continuous pipeline of levels and plot, and a pipeline is a commitment rather than a task.
We ended up with a second field next to D. Three values, no numbers: one person can do this, one person can do this badly, one person cannot do this. The middle value turned out to be more useful than the score, which surprised me. A 7 does not mean impossible. It means you will ship a reduced version of the genre, and you should decide up front whether the reduced version is still the thing that got validated, because very often it is not, and you find that out late.
Two things I got wrong
I had a prior I was certain about, which is usually a good sign to go check something. Easy to clone means easy for everybody, so the easiest genres should also be the most crowded. Obvious. I computed the correlation between difficulty and crowding across all fifteen rows of our current snapshot and got −0.018, which is nothing at all, and I sat there for a while hoping I had typed something wrong.
Fifteen rows is a small sample, and these are my own subjective integers rather than measurements of anything, so I would not defend that figure past one decimal place, and maybe not even that. What I would defend is the direction. My obvious prior has no support in the data I actually have.
The band structure is more interesting than the correlation, and it is not symmetric, which is the part I think about now. Of the five rows scoring 3 or less on difficulty, four sit at 7 or above on crowding, and the mean for that band is 7.4, with sorting puzzles, line drawing puzzles and stress relief toys all arriving at D = 3 and S = 8. Being easy to build does tend to come with a crowd. Being hard to build tells you nothing whatsoever, because in the band scoring 6 or higher the crowding values run from 3 to 9 with a mean of 6.7, a spread so wide that it is not information. No protection there. The easy end of the scale is predictable and the hard end is a lottery, which is backwards from how I had been reading that column for months.
The second result is the one I liked less. Difficulty correlates with our domestic validation score at +0.63. The genres hardest for one person to build are also the most proven at home: the two rows carrying a difficulty of 9 have validation scores of 10 and 9, which are the two highest values anywhere in that column.
That is not a coincidence. I should have seen it coming. Validation at that scale is evidence of a large operation standing behind the genre, because a mechanic does not climb to the top of a national chart by being small, and the machinery required to put it there is the same machinery that makes it unbuildable for one person.
Which means our difficulty veto, the rule we wrote to protect ourselves from impossible projects, fires precisely on the rows the model finds most attractive. I find that genuinely uncomfortable to sit with. It is correct, and I am keeping it, and I still do not enjoy it. The rule is not really filtering the market at all. It is filtering out the part of the market that was built by companies, and that turns out to be most of the validated part.
Low difficulty is not a gift
There is a trap at the other end too, and it is subtler, because a low score feels like free money. Retro casual scores 2 on difficulty. It also scores 7 on crowding, and the note on that row says the quiet part out loud: trivially easy to make means trivially easy to copy. Nothing about a push box puzzle belongs to you. If the entire case for a genre rests on being able to build it quickly, then the case expires the moment somebody else builds it quickly too, and there is no version of that race you win by being first by a week.
Crowding is not the only thing that shows up alongside a low score. One row in the table carries a validation score of 5, which on our own published basis means “not mainstream domestically”, so the whole argument for it rests on a guess about demand in a market we have never tested, while the difficulty score of 4 sits there quietly making the row look safe. An easy build in an unvalidated category is easy to start and hard to learn anything from, and learning something is the actual point of shipping it.
The check that exposes all of this
What makes me trust any of the above is the weight test. Our default weighting is a judgement call, so we re-sort the table under three other weightings and count how much of the top five survives. Under a domestic first weighting, five places out of five hold. Under an export first weighting, three. Under a shipping volume weighting, which pushes the difficulty term up to 0.40, only two hold, and a line drawing puzzle climbs into the top five purely on the strength of being easy.
That last reshuffle is the most useful result in the whole exercise. When I weight difficulty heavily, the table starts recommending the genres I can build rather than the genres with room in them, and it does this so smoothly that the output looks identical in format to the honest sort. It is measuring my own preferences and reporting them as market information. Once you have watched that happen you cannot unsee it, and you stop reading the composite as an answer to anything.
The practice I have settled on is to fill D in first, before I look at validation at all, which led us to audit the whole model. Validation is the exciting column. It moves. If I write V first and then assign D, I already know which answer I want, and I will find a way to argue myself from an 8 down to a 5 without noticing that I am doing it. Writing the boring number first, while the row is still uninteresting, is the only trick I have found that survives contact with a genre I would enjoy making.