The dead-end rate was always zero

Clarify's design document specifies dead-end rates of 15 to 35 percent for the difficulty bands. Every level in the daily pool has deadEndRate set to zero. The generator never computed it. This is the gap between what we designed and what we shipped.

When I first ran verify.js I expected to see a distribution. The design document section 2.0 explicitly ties bottle capacity to difficulty. Unequal bottles create a bin-packing problem that produces dead ends. With equal capacities, the greedy strategy always works and dead-end rates sit near zero. I wanted 15 to 35 percent, not zero.

The shipped levels tell a different story.

Par distribution across 399 daily Clarify levels. All levels show deadEndRate=0 despite design expecting 5-40%.
Par distribution across all 399 daily levels. Every level has deadEndRate = 0, while the design wanted 5-40% depending on the tier.

Two measures of difficulty

The solver runs as part of the verification step. It proves par for every level using bounded breadth-first search. I extracted the deadEndRate values from all 399 daily levels. The maximum is zero. The minimum is zero. All of them.

The dead-end rate measurement requires a probabilistic probe. From the initial state, take 200 random legal pours. For each state after a pour, run IDA* with a fixed budget. The ratio of states that cannot be solved within budget is the dead-end rate. This is a probabilistic measurement that samples 200 random legal pour sequences from each level state, then applies an IDA* solver with a bounded node budget to determine which sequences reach a solvable state versus which sequences create states that cannot be solved within the allotted computation budget, yielding a percentage that represents the likelihood of hitting a dead-end path during normal play, and this measurement is what the design document expected to see across the difficulty bands.

The tax is a different measure of difficulty that reveals where the real complexity lives in a puzzle design, and it is the gap between the floor and the solver's answer. The floor is a lower bound you can read off the board. Count liquid layers, subtract colors. That is the minimum number of moves any solution needs. The tax measures how much the optimal solution exceeds that bound, representing the real decision depth needed to actually solve each level, and the tax also correlates tightly with verification cost because the solver must explore more state space when the gap between floor and optimal grows larger, and this correlation tells us where the actual difficulty hides in the game's design.

Why the design wanted dead-end rate

The design argument is precise. Density-based settling makes separation free because light liquid always floats on heavy liquid and can always be scooped back. If every bottle has the same capacity, parking a color costs nothing. You pour the lightest layer into any bottle with space. The greedy strategy always works. Optimal moves stay at two to eight. Dead-end rates sit near zero. There are no real decisions in a level where any greedy choice leads to a solved state.

Making bottle capacities different prices parking. A color must return to its own bottle if that bottle has room. This turns parking into a bin-packing problem where wrong choices create dead ends. The design's Appendix B shows prototype data: equal capacity gives 0-4% dead-end rate, unequal [6,5,4,3] gives 20%, and [7,6,6,5,4,3] reaches 54%. The target range of 15-35% dead-end rate requires unequal capacities to be enforced.

Where the bug lives

The capacity decision is the bug. The layout code supports unequal capacities. The renderer supports it. The core rules support it. The solver supports any capacity array. Only the generator was switched to uniform caps, and the dead-end rate computation was skipped entirely.

The comment above the TIERS array says uniform capacity keeps every board's tubes equal-height and every unit of liquid the same visual height, matching the shipped Clarify design. Visual equality was prioritized over difficulty tuning. The visual layout works, but the difficulty band misses its mark because dead-end rate is always zero.

What the numbers show

I measured across all 399 levels to confirm the discrepancy. The par distribution spans 2 to 9 with an average of 5.17. The solveNodes range from 28 to 575365, with a median of 20020. The distribution of the gap between the floor and the solver's answer climbs with par. Some levels take only a few hundred nodes to prove. Others require hundreds of thousands of nodes. The 68 levels with a tax of 3 take a median of 108626 nodes, about 74 times more than easy levels. This is the real difficulty hiding in the levels that we ship.

The dead-end rate, which should have been 15-35%, is exactly zero across the entire pool. That is why the tax stays small across all levels. Without the dead-end traps, wrong moves do not create unwinnable states. Players cannot make an irrecoverable error. This is a feature, not a bug, if your target is pure relaxation. But it is not the difficulty feature the design argued for.

Two days that hit the upper bound

Seventeen levels sit at par equal to the documented upper bound for their tier. GEN-fri-5783 is the only par-9 level at 342063 solver nodes. GEN-sat-5722 reaches 233937 nodes at par 6. These are the exceptions that prove the rule. The rest of the pool is below the designed difficulty.

What this means for players

The daily difficulty gradient still exists, just not in the way the design intended. Par still increases through the week, from Monday's average 3.26 to Friday's 6.26. But the key design lever for creating meaningful decisions, dead-end traps, does not exist in the shipped game. Players cannot stumble into an unwinnable state, so there are no second chances on bad moves. This is relaxation-first design, whether intentional or not.

The fix is five lines

The TIERS array in tools/gen_levels.js would need to change to implement unequal capacities matching the design. Monday's [6,6,6,6] becomes [6,5,5,4], Tuesday's [6,6,6,6,6] becomes [6,6,5,5,4], and so on. The canFitColors check already handles unequal caps, the scrambler works with any capacity array, and the solver proves par for any capacity array. Only the generator settings need updating.

Regenerating the 399 levels takes a few minutes on a laptop. The daily snapshot in tools/daily-snapshot.json needs updating. The deployment would then ship new levels for every future date while past dates keep their frozen levels. The new levels would have non-zero dead-end rates and would actually challenge players to make the right decisions at the right time.

The discrepancy between the design document and the shipped generator is measurable and documented in the code itself, and any developer can verify the numbers by running the same analysis I have run here, showing that the generator comment is incorrect and that the shipped levels do not match the difficulty target of the design, which is why I am writing this article to make the gap visible to anyone who cares about the actual data that the solver produces when it proves each level's optimal move count, and this visibility is essential for maintaining the integrity of the measurement pipeline that the design document depends on for its difficulty bands.

We have not made the change because the generator comment says matching the shipped design, but the design document says something different. One of them is wrong. The numbers are in the repository and the solver proves them every time node tools/verify.js runs.

← Back to the blog