What 10,000 puzzles actually need

The short version

We generated 10,000 puzzles and graded every one with the same engine that rates the boards in this site's game and practice sheets. Four things came out of it that contradict what most guides imply.

The difficulty label describes a technique, not a difficulty. Every puzzle the game gives you at Easy was solvable with singles alone — 600 out of 600. Every one at Expert needed a chain or a swordfish — 600 out of 600. The five settings are not a smooth ramp. They are five buckets, and the grader puts a board in exactly one.

X-Wing is rarer than the guides admit. It appeared in 3.5% of puzzles dug to 26 clues. Skyscraper appeared in 10.1% — nearly three times as often, while sitting after X-Wing in almost every teaching order. At the Hard setting, 9 puzzles out of 600 needed an X-Wing; 591 needed something harder.

Three techniques never showed up. Hidden Triple, XYZ-Wing and Jellyfish appeared zero times in 5,000 puzzles. They are real, they are teachable, and they are not something you will meet by accident.

Clue count is a weak predictor. Among puzzles with 26 clues, 58% needed nothing beyond singles and 11% could not be finished by any of the sixteen techniques. Same clue count, wholly different work.

How this was measured

The grader is the same code that rates every board in the game. It solves a puzzle by applying techniques from cheapest to most expensive — singles first, then pairs and box-line, then X-Wing, then the skyscraper family, then chains — and records the highest-level technique the solve actually needed. A puzzle it cannot finish with all sixteen techniques is marked "no technique finishes it" and counted, not discarded.

Three samples were taken, all with the same seed, so the run is repeatable:

• 3,000 puzzles generated through the game's five difficulty settings, 600 each.

Graded after generation, so the number below is what the setting produced, not what it claims to produce.

• 5,000 puzzles dug to exactly 26 clues with no difficulty target at all. This is

the natural frequency sample — nothing was filtered to make a technique appear.

• 2,000 puzzles spread across eight clue counts from 22 to 36, for the last table.

What each difficulty setting actually gives you

The Easy and Medium rows are what you would expect. The Hard row is the one worth reading twice.

A puzzle is rated Hard when X-Wing is the most it needs. But X-Wing is scarce: of the 600 boards the Hard setting produced, only 9 topped out at X-Wing. The other 591 needed a skyscraper, an XY-Wing or a unique rectangle — techniques that most guides place in a later chapter.

This is not a bug in the setting. It is what the supply looks like. A board that is solvable with singles, pairs and X-Wing, and nothing beyond, is genuinely uncommon; when the generator digs to a clue count that requires a named pattern, the pattern it finds is usually a more common one. The setting accepts either, and the common one wins by sheer frequency.

What it means at the board: if you can spot an X-Wing but freeze on a skyscraper, Hard will feel unpredictable. That is not you missing something. That is the distribution.

The Master row is the other oddity. Every one of those 600 boards defeated all sixteen techniques. "Master" here means the grader cannot reason its way through — the puzzle needs a guess, a chain you build by hand, or a technique this engine does not implement. It is a real category, but it is not "the next step up from Expert". It is a different kind of wall.

How rare is each technique

5,000 puzzles, each dug to 26 clues, no other filtering. A technique counts if it appears anywhere in the solve path — not only as the hardest step.

Read the bottom of that table first. Hidden Triple, XYZ-Wing and Jellyfish: zero, zero, zero. Not rare. Absent. If a guide spends a chapter on them without saying so, it is describing a puzzle that essentially does not occur at this clue count.

Swordfish is barely better at 0.3% — 16 boards in 5,000. This site has a swordfish practice page, and building it took a much larger pool than the other five, because the supply really is that thin.

Then look at the order. Skyscraper (10.1%) is roughly three times as common as X-Wing (3.5%), and XY-Wing (8.2%) more than twice. Almost every teaching sequence puts X-Wing first because it is the simplest idea to explain — one digit, four cells, a rectangle. Simple to explain and common to encounter are different properties, and the standard order optimises for the first one.

That is not an argument for teaching the skyscraper first. X-Wing is still the right introduction to single-digit patterns; it just is not the workhorse its reputation suggests.

Does fewer clues mean harder?

No, and the table below is the reason. Each row is a separate batch, all generated the same way, differing only in how far the puzzle was dug out.

The trend exists — 22-clue boards need named techniques about 40% of the time, 36-clue boards about 2%. But the trend is noisy enough to be useless as a difficulty measure. At 26 clues, the most common outcome is still "singles only". Two puzzles with the same number of givens can differ by four engine levels.

The reason is structural. Digging removes cells; it does not choose what the remaining board requires. Whether a puzzle needs a skyscraper depends on which cells happen to be left, not on how many. Clue count describes the size of the hole, not the shape of the work.

This is also why the game rates every board after generating it instead of trusting the setting. The setting chooses a target; the grader decides what came out.

What to do with this

If you are choosing a difficulty: treat the setting as "which technique will I be practising", not "how long will this take". Easy is a singles drill. Medium is pairs and box-line. Hard is the skyscraper family with the occasional X-Wing. Expert is chains. Master is a puzzle the engine cannot finish.

If you are learning techniques: the two rarest rows — swordfish and the three zeroes — are not worth your time until you can spot a skyscraper and an XY-Wing reliably. Not because they are hard, but because you will not meet them. Practise the common ones and the rare ones will show up on their own, eventually.

If you are reading a guide that orders techniques by difficulty: check whether the order is about conceptual simplicity or about frequency. Most are the former, and they are usually honest about it. Just do not mistake "explained third" for "appears third most often".

What this data does not prove

The samples are generated, not collected from published puzzles. A newspaper's "hard" is not this site's "Hard", and nothing here tells you how a specific book or app distributes its techniques. The numbers describe what this generator produces at these clue depths.

The 5,000-puzzle sample includes roughly 11% boards the engine could not finish. For those, the solve path stops where the grader got stuck, so techniques it would have needed later are not counted. That biases the rare end of the table slightly downward — which means the true frequency of swordfish is if anything a little higher than 0.3%, not lower.

Single-run estimates carry sampling error. At 5,000 puzzles, a technique appearing 3.5% of the time has a margin of roughly ±0.5 percentage points. The zeroes are the one result that sampling error cannot explain away: three techniques, 5,000 puzzles, no occurrences.

The raw data

Everything is in one file, including the per-sample breakdowns used for the tables above.

• stats.json — every count, the seed, and the sample sizes
• technique-rarity.csv — the frequency table on its own

Re-run it yourself with node tools/gen-stats.js; the seed is fixed, so the numbers come out the same.