Dire warnings from crackpots
Another AI guy has gone rogue while coincidentally building his personal brand. Jacob Coxon has resigned from Anthropic because, as he says on X, the everything including pedophilia app, the frontier labs aren’t “acting responsibly” and are “gambling with our lives.” After making lots of money from this as an employee of both OpenAI and Anthropic, Jacob can’t take it anymore.
I never really know how to respond to things like this because there are many valid reactions, some of which even initially seem to be contradictory. Despite warning against AI, Coxon is making the exact same sorts of bold yet ill-defined predictions that AI boosters do. He says we’re “racing straight to self-improving superintelligence,” which is dubious regardless of whether you believe this is good or bad — and anyway, still elides the basic point of what the hell “superintelligence” really means. Not that the guy really ought to pause in a Twitter thread to define this, but somebody, somewhere, should.
He also claims that “the people building AI earnestly believe that it could kill us all by the end of the decade.” That they believe this, I have no doubt. But could it actually do that? How? I’m not asking rhetorically. What are the mechanisms by which even a self-improving superintelligence could kill us all? What could we do to stop them?
Like all of the most lurid predictions about AI, I would bet that Silicon Valley crackpots see the extermination of (most of) humanity as a feature and not a bug. Imagine a system so powerful that it could end life on earth. Now imagine that you, a dork-ass loser with a punchable face, have created that system. What does it make you? It makes you God, right?
I don’t mean to completely rule out the possibility of AI causing a mass casualty event. Just because something is stupid doesn’t mean it can’t also be dangerous. Some of our biggest dumbasses have been responsible for some of our most painful tragedies. There’s really no reason to think that what we’re calling AI can’t wreak havoc with the best of them. It’s just that it’s probably less “evil genius” stuff and more “dog that’s been trained to run between tank treads with a bomb on its back.”
Still, there’s one more point to make about relative risk. Coxon asserts that “no other human activity poses this level of danger.” That’s an ironic thing to type while inhaling wildfire smoke, but it’s emblematic of the SV way of thinking. Hypothetical future risks are far more pressing than the risks that we are facing today. The people who think it’s easier to make the planet Mars habitable than the planet Earth will of course think that their superintelligent terminator machines are more dangerous than climate change, pandemic, war, and all the other blights presently afflicting us. You have to remember that, in the end, these guys just aren’t very smart. Or maybe it’s more accurate to say they’ve disappeared all the way up their own assholes.
And they’re dragging the rest of us with them.