This week bestie Luce had the scoop on an announcement from the UK government that they were giving tech companies three months to prevent children from sending and receiving naked images on their phones and tablets. The requirement would apply to existing devices as well as new ones, and the Home Office says it would make Britain the first country in the world to attempt protection at this level.

This is one of those announcements that sounds really amazing as a political headline, and then you stop to think about what’s involved in making it work, and so I took some time this week to dig into the details. Regular readers will know I’m not really a fan of most of the attempts we’ve made so far to protect young people online. I wrote at length about the UK’s approach to online safety, and the age-gated internet in general, here.

When governments or platforms decide what counts as “adult,” they rarely mean straight, white, or heteronormative depictions of desire. Age-verification regimes risk hard-coding those same biases into the architecture of the web, making marginalised people once again the test cases for what’s considered acceptable to see.

And this new UK announcement landed in the middle of more of the same: it came packaged with new screen-time guidance for 5-16 year olds, while the government there sifts through 116,211 responses to its consultation on an Australian-style social media ban for under-16s — with a similar ban considered imminent.

But I want to take this new proposal seriously, because it's a much more focussed and specific idea, and it’s worth understanding what it actually means.

Here's the case for it. Unlike "keep teenagers off instagram," which treats the entire social internet as a contaminant, this measure targets a documented, escalating harm. The Internet Watch Foundation reported that child sexual extortion cases in the UK rose 72% in a single year — criminals tricking young people into sending nude images, then blackmailing them. In 97% of those cases, the victims are boys. The Home Office says 91% of online child sexual abuse reports in 2024 involved self-generated content. There are now cases of ordinary photos from school websites being altered with AI to create abuse material, and the schools themselves blackmailed. Several children caught in sextortion schemes have died by suicide.

This new announcement then is not the same as a moral panic about screen time. Photos are a mechanism of abuse, and the proposed intervention sits at the exact point in the chain where it happens: the image leaving the device.

One of the confusing things about the announcement was the UK saying this technology “already exists” and that Apple and Google just need to turn it on. That seemed weird to me, so I wanted to get into it further.

It’s true that Apple's Communication Safety feature already detects and blurs nudity in Messages for child accounts, on-device. A friend this week said that his daughter had triggered this by sending a screenshot of a janky free game to a friend that had a porn ad in the corner, for example. He and his wife both received alerts.

Now, you might be thinking, “didn’t Apple already do something like this a while ago and roll it back?” (if you, like me, have a memory that’s mostly sieve). And you’d be sort of right. In 2021, Apple announced child safety initiatives that covered two different things. One was a scan of all photos uploaded to iCloud against a database of known CSAM images, provided by the National Center for Missing and Exploited Children. The second was the Communication Safety measures in messages. These two things got muddled together, and many people thought that Apple was now basically looking at every one of your photos all the time. (It wasn’t)

So it’s not really as simple as the technology exists and just needs to be “turned on”, but it is true that tech companies can, in theory, tell what photos we have are naked. It’s just that to do that we have to decide how much of our privacy we’re willing to concede, and in what circumstances.

And your first question needs to be: how does a device know it belongs to a child?

For a phone to block nude images "for children," every phone has to establish whether its user is a child. Which means age assurance for everyone. Buried in the coverage is the acknowledgement that all adults will need to verify their age if they want to take or view nude images on their own devices. The age checks I wrote about in ID, Please lived at the platform level — a bouncer at the door of each nightclub. This proposal moves him into the operating system. The checkpoint stops being something you encounter when you choose to visit certain places online and becomes a property of the device in your pocket. You’ll have to prove you’re an adult to own and use a phone.

The pushback was pretty immediate. Signal published a statement titled Surveillance Is Not Safety, arguing that scanning all content on all UK devices on a presumption of nudity endangers everyone while handing Apple, Google and Microsoft even more control over our most personal information. Their core warning is about scope: once the infrastructure exists, its targets are defined by whoever holds power — "nudity today and political speech tomorrow." They argue:

Child safety looks like well-funded education, robust social services, and meaningful guardrails on the very AI technologies and platforms the current government is eagerly courting. What the UK government wants instead is invisible surveillance infrastructure, switched on by default and potentially rushed into law under cynical pretexts.

Big Brother Watch's Silkie Carlo also predicted the practical outcome: population-wide identity checks just to use a phone, tablet or laptop.

When I read things like this, I get a bit frustrated, because part of me is always like “surely we can agree on kids not getting sextorted?! Maybe it’s okay to give up a bit of privacy for that?” — But then, the flipside is we know we can’t trust big tech or government to stay in their lane.

In 2021, a group of the world's most senior cryptographers and security researchers published Bugs in Our Pockets, a comprehensive analysis of client-side scanning. Their conclusion was that it makes citizens searchable on an industrial scale, and that even a system built for clearly illegal content would face irresistible pressure to expand. Nudity-detection systems that report to parents can out LGBTQ+ kids before they've chosen to come out, and can alert abusers that an intimate partner is sharing images with someone else. A system that merely reveals the existence of an image can do real harm, depending on who it reveals it to. The history of "protecting children" online is littered with measures that ended up endangering the most vulnerable people they touched.

When Apple rolled back its CSAM scanning feature, its privacy chief warned that scanning would create new attack surfaces and a slippery slope of unintended consequences.

So where does that leave us? Genuinely torn, I think, is the honest place to be.

There is a narrow version of this proposal that’s worth having: on-device nudity blocking, default-on for verified child accounts, no reporting to anyone, no scanning of adults' content, no identity infrastructure for the rest of us. Basically what Apple's Communication Safety already does. That version addresses a real and horrifying harm at the point where it occurs.

I think where I land is that after years of discussing age-gating the entire internet and threatening to ban teenagers from the social web wholesale, a proposal aimed at the specific mechanism by which children are actually being exploited is the closest thing to a real idea this debate has produced so far. I guess the next three months will reveal more about the motivations of all involved.

more good stuff

• in just over an hour I’m talking with bestie Mike about his new book. It’s FREE, and it will be fun. Come join us.

• following on from our convo about the indie game Mixtape, they’ve released this super-cute “make your own mixtape” site that does just that.

• incredible watch - bestie Adam meets the last traditional hand engraver in San Francisco - Marlen Hazel.

• because everyone keeps asking me, yes the Utah lego scandal is worth your time — it’s absolutely unhinged, get popcorn. Here’s a summary.

finally, in my lego city

Forward this email to someone with a mixtape.