Technology

Fair point, and I'll concede, but only partially, as it is nit what is being discussed here. Starmer's proposal isn't asking for Apple to expand their system, it's mandating platforms to comply and make it impossible, and the platforms can choose how to do so. On-device detection that never leaves the device is a meaningfully different privacy profile than server-side scanning or breaking E2E encryption. Apple's Communication Safety feature works roughly as you described and that architecture is less invasive than the worst case scenario. If every implementation were genuinely on-device, opt-in, parent-controlled, and open source verifiable, it would be a different conversation.

But that's where my concession stops.

We can only take a corporation's word that it's truly on-device and nothing is retained. The history of that promise is not encouraging. There have been multiple instances across the industry of companies guaranteeing on-device processing only for that data to appear in breach disclosures afterward. Closed, proprietary systems cannot be independently verified. We're being asked to trust the architecture of companies whose entire business model is built on data extraction.

There's also a false positive problem. Google has already implemented similar detection and there are confirmed cases of users having their entire Google accounts permanently locked after photographing their own child in the bath. Emails, photos, Drive, business files, income streams, all gone, with no meaningful appeals process. The harm from a false positive in a system like this isn't a minor inconvenience, it's potentially catastrophic and irreversible.

And then there's the infrastructure problem. The Patriot Act is, once again, the prime example. You build the architecture for one stated purpose and then it gets legislated into something broader. Age verification is the live example happening right now. It started as self-attestation. That wasn't sufficient so it became on-device ID verification. That wasn't sufficient so it became third party trusted providers. Private vendors like Persona and kID. Both of which have had documented breaches after promising on-device verification themselves. This is literally the documented trajectory of every surveillance infrastructure built in the name of protection.

It's never a matter of if they legislate it further. It's when. And who profits from the expanded version.