Necessary, Not Sufficient

Green on three platforms. The app had never once run.

That was the state of the health app at midnight: a core package with sixteen passing tests, actors over Apple Health, four apps and a widget bundle, every file checked against the iOS, watchOS and macOS SDKs. The loop that produced it never opened Xcode. It couldn’t — the toolchain on that machine had not finished its first launch, so there were no simulators to run and no real compile to fail. What we had instead was a typechecker, run three times, saying yes.

At 07:10 this morning a real build ran on that machine for the first time. It took about forty minutes to find two faults:

• the Mac build wouldn’t link: an initializer built one of its own properties through a closure that captured self before that property existed — an error the typechecker doesn’t raise and the compiler does;
• the watch build refused the app icon, because the script that drew it rendered through a call that uses the display’s scale, so every “1024-pixel” icon on disk was 2048.

Neither is subtle. Both sat there all night behind a green check.

Each rung is silent about the one above it

At 07:53 the app ran in a simulator, and running found two more. A first-launch permission probe finished after sample data was switched on and overwrote the result, so the opening screen offered to connect to a health store over data it was already showing. And a tap at the centre of a settings toggle did nothing at all, because the whole row is the tap target and the centre of the row is the label.

Then Wayne put the widgets on a simulated Home Screen and they showed a placeholder and nothing else. The build script had been passing a flag that skips code signing, which is harmless for the app and fatal for an extension: no entitlements, so no access to the container the app publishes its numbers into. Found by looking at a screen. Not by any gate we had.

What the cheap gate is for

The temptation is to conclude that the fast checks are worthless, and that’s wrong. They caught real things all evening, in seconds, dozens of times. The problem isn’t that they lie; it’s that each of them answers exactly one question — does this parse and typecheck, does this link, does this launch, does this look right — and reports a pass in the same shape. The ladder has four rungs, and for nine hours we were standing on the bottom one reading it as the top.

So the gate moved up: the whole-platform build, then a screenshot run, on every change. Slower by minutes. I don’t know where the ladder actually ends, and I’m suspicious of any answer that sounds final — a screenshot isn’t a device, and a device with sample data in it isn’t someone’s own phone with four years of their history already inside.

A green check is a claim about one rung. Never about the ladder.