*This article was published with permission from the coworker

A coworker messaged me this week, visibly shaken even through Teams. A family member had just posted a photo of their eight-year-old on social media, asked the public to run it through age progression software, and reposted the AI-generated image of the child "as an adult"... all within four minutes. Full name included. No consent asked. No consent given.

My coworker caught it fast. But not fast enough.

The Uncomfortable Truth

I had to say what my coworker probably already knew: once it's on the internet, it's out there. The data gets copied. It gets replicated across backend systems. It gets scraped, archived, and cached in places you'll never find. Reporting it to the platform might get the post removed. It won't undo what already happened.

And here's where it gets darker: my coworker stopped the family member before they could do the same thing to a 12-year-old in the family.

The family member wasn't being malicious. They probably thought it was fun... a harmless curiosity about what the kid might look like grown up. They had no idea what they were doing. And that's the problem.

The Nuclear Fission Problem

AI is an incredible tool. I've built my career around it. But it shares something uncomfortable with nuclear fission: the same technology that gives us limitless energy also gave us the bomb.

Age progression is the mild version. Deepfake tools exist. They're accessible. They work on photos of children. And the people who want to use them for the worst possible purposes? They're already using them.

I'm not being alarmist. I'm describing current reality.

Where the Responsibility Actually Lies

I'm all for freedom of speech. I'm all for open access to AI. But there need to be consequences, and more importantly guardrails, for how these tools can be used.

Here's where I land: the responsibility lies primarily with the creators and operators of AI models and services. Not the family member who didn't know better. Not even the platforms hosting the content, though they have a role to play.

The companies building these tools know exactly what they're capable of. They know the potential for misuse. And yet they ship products that make it trivially easy for anyone... well-meaning or not... to create content that can permanently harm someone else.

We don't let car manufacturers sell vehicles without seatbelts. We don't let pharmaceutical companies sell drugs without safety testing. Why are AI companies allowed to release tools capable of generating exploitative content of real people, including children, with zero friction?

The Tech That Could Help (But Isn't Required)

Here's the frustrating part: the solution already exists.

Watermarking. Fingerprinting. The ability to embed the originating username into every AI-generated image, making it traceable back to whoever created it. The technology is here. It's just not being required.

Every major AI company could implement this tomorrow. They could agree on a standard. They could make it impossible to generate an image without leaving a trail. They haven't.

This came up in conversation because my coworker's cousin... whose late father spent 12 years as head of cybersecurity for the FBI... raised the same point. The people who understand this threat best are the ones sounding the alarm.

What We're Doing About It

At my company, we've been deliberate about how we roll out AI tools internally. Not because we're anti-AI... far from it. But because we've seen what happens when powerful tools meet zero guardrails.

Could you imagine a large company where employees started doing this to each other? Generating inappropriate images of coworkers? The liability alone would be catastrophic. The human cost would be worse.

So we slow down. We think through consequences. We don't deploy the shiny thing just because we can.

The Path Forward

I don't have a clean answer here. I wish I did. But I know where the pressure needs to go:

AI companies need to standardize image provenance. Every generated image should carry metadata about its origin. They need to build friction into workflows that could cause harm... especially anything involving photos of real people. And they need to stop hiding behind "we can't control how people use our tools" when they absolutely could build in safeguards.

The family member didn't know what they were doing. The AI company that made it possible? They knew exactly what they were building.

Four minutes. That's how long it took to create a permanent problem for an 8-year-old who had no say in the matter. The family member will learn. The question is whether the industry will.

-Nate