A cursory examination of our recent AI-related news is playing out like the opening montage of a bad apocalypse movie—and cut to an abandoned highway littered with burned-out cars and wildlife retaking the cities. Actually, the most likely end-of-the-world scenario making the rounds is that the most efficient way for AI to power itself is to superheat the planet past the boiling point of water. Oh shit, did I just give it an idea? Man, it’s so easy to screw this up.
I used to say, when people asked me what I thought about AI, that it would erase history. I meant that people would use it to erase history; of course they would, and they are. People will use it to do all kinds of terrible things, and I still think that is the biggest threat in front of us. But I can also put my problems with developing artificial general intelligence (AGI), or artificial superintelligence (ASI), into two general buckets. The first is how it’s being developed, especially in the West. The second is the larger, more philosophical conundrum.
Let’s start with how it’s being developed here in the West. I started saying Nationalize Silicon Valley long before the AGI problem captured headlines. What we think of as Silicon Valley had already become a group of super-monopolies who at the time had seemed, and now have been proven to be, an ever-increasing threat to the fabric of our society and to our hold on representative democracy. Since the growth of OpenAI and Anthropic and the whole AI arms race, nationalizing them has only become more integral to our survival. And to clarify, by nationalize I mean antitrust stuff like breaking up the monopolies and putting much of the infrastructure under heavy government regulation like any other necessary utility or industry—or even better in the case of the internet, a public commons of sorts. Then we need to take the truly dangerous stuff, like the development of AGI, and get it into some sort of international safety framework. After that, we can regulate the resulting smaller companies in a way that’s healthy for society, so no more predatory business models. Of course, we can’t do any of that until we are safely in some semblance of a functioning democracy, which clearly we currently are not—and that is in no small part thanks to Silicon Valley itself.
We need to talk about developing AI in a way that is akin to nukes, with tight security and government control, and then quickly developing something like the Nuclear Regulatory Commission to oversee development with transparency and oversight. Of course everyone’s response is that we can’t trust the government. Well, we can’t trust corporations either, so take your pick. It should be obvious that letting private companies develop AI with almost no regulations is pure insanity, and the nuclear comparison illustrates that. We didn’t let DuPont or Union Carbide or M.W. Kellogg develop nuclear weapons in the free market and let them claim that regulating them would ensure that we lost the arms race against the Nazis or then the Soviet Union. But that’s exactly what we’re doing today by letting Meta and Google and Anthropic and OpenAI develop a technology that is promising to be much more threatening to human survival than nuclear weapons. It’s absurd, especially when you realize that means Mark Zuckerberg, Sam Altman, not to mention the likes of Musk and Thiel and all the other whack jobs who are now spending trillions to develop a technology that they refuse to develop safely, and in general aren’t even sure how it all actually works. I’ll get back to that last point in a minute, but for now I just want to say that this whole scenario is crazy, and if I sound crazy just saying so, well, then color me crazy.
A quick side note: I’m comparing Silicon Valley’s method (or madness) with China because that’s the other player spending the most money. Many nations are working on this, and while everyone is reliant on global supply chains and research, China is spending more than every other country combined. And Silicon Valley is outspending China by some estimates as high as 8-1.
So anyway, in China, AI is being developed more or less under a much stricter, government-controlled open-source model that they are promising to share with the global South. Not to say that it isn’t dangerous, but at least there are some guardrails. I don’t want to get too sidetracked into this comparison right now, but I thought it was worth mentioning. Because until the OpenAI hacking incident, the only guardrails being thrown up around Silicon Valley were weak legislation coming out of Europe, never gonna happen under the MAGA regime musings in democratic circles in the US, and some consumer/market based ideas coming out of people who cut their teeth in Silicon Valley itself. There are a slew of think tanks and organizations trying to find ways to align AI development with the values that would ensure that the technology best serves the interests of humanity. This problem of alignment has been popularized by people like the philosopher Nick Bostrom and the AI researcher Eliezer Yudkowsky—Yudkowsky co-wrote the book If Anyone Builds It, Everyone Dies. So now there are Ideas coming from places like the Meaning Alignment Institute, for instance, which is proposing ways to align AI with markets, democracies, and what people value in what they call the full-stack alignment. And that’s great, I agree with that, except for the markets part, because as I’ve argued plenty, the market is not aligned with democracy or the health of the planet, but in general I like the idea of aligning technology with our values. In fact, I will say that our inventions, our technologies, what we invest in, are generally a reflection of our values. That’s what makes Joe Edelman’s, co-founder of the MAI, statements about training AI on our values so important and also so out of step with what’s being done in Silicon Valley. To paraphrase Edelman, if AI is going to serve us well, it’s going to need to get us. You know, like really get us. It’ll need to understand our values and our norms and what’s important to us and all that jazz. And that makes sense. Will that ever be possible? The Meaning Alignment Institute thinks so, and they have a strategy for that, and many other people feel it’s possible as well. Me, I’m not sure, no surprise there. And as Bostrom and Yudkowsky keep pointing out, whose values? Or do we, or can we, even understand our own values? Right?
Anyway, there are plenty of others talking about instilling some sort of alignment with the powerful AI systems that are coming our way. Geoffrey Hinton, often called the godfather of AI, has said that we need to develop AI to ‘like’ us more than it ‘likes’ itself, so that its goals are aligned with us realizing our full potential. Another pioneer of AI, De Kai, who wrote the book Raising AI, uses the metaphor of ‘parenting’ to instill our values, especially empathy. He argues that we’ve been interacting with a form of AI for decades and are already influencing each other without us being mindful enough in that process. The parenting metaphor could help make Artificial Intelligence a beneficial cohabitant of a world where we can continue to learn from one another. De Kai has said in recent interviews that when we achieve something like AGI, we will likely be the last generation to have the ability to be parents, and then, for survival’s sake, the most logical step may be to merge with technology or be left behind. Makes sense, but not really my bag. I’m into more earthy systems, and I’ll probably be left behind. So I get the argument that anything we create is as natural as anything else in the universe. A beaver dam is as natural as the beaver. Right? Good question, worth exploring. Well, this is all interesting stuff, and there are lots of people working in this realm. And in my mind, definitely something beneficial could be possible once we wrench the development of AI from Silicon Valley. But even then, I’m thinking that we’re going to need to move away from trying to develop AGI altogether—regardless of whether it’s even possible. So this may be a good time to explore what Silicon Valley will create if we don’t stop them.
If our values inform our inventions, then what can we expect from whatever is developed by the likes of Altman, Zuckerberg, Musk, and all of the rest? They stole all of the intellectual property pretty much ever developed to train their models. Their core mantras are still move fast and break things and catch me if you can. They’re buying local governments across our nation and the world to build unregulated data centers. They’ve abandoned Climate Change and instead have spurred China into a reckless and unwinnable arms race. As Karen Hao, author of Empire of AI, has pointed out, they’re employing vast numbers of people in some of the poorest places on earth to label, annotate and ‘clean’ data that is used to train AI systems. This work often means watching an endless amount of damaging content for extremely low wages. Antonio Casilli, author of Waiting for the Robots: The Hired Hands of Automation, estimates that 160 million people are involved with this sort of low-wage data work. And as the Nerd Reich’s Podcast Host Gil Durán is endlessly pointing out, much of Silicon Valley’s business models are based on deception, exploitation, surveillance and outright theft. So, if our technology is a reflection of our values, what can they possibly produce over there that would benefit humanity? And to De Kai’s point, the leaders in Silicon Valley sound like really shitty parents. They’re also the most dangerous monopolies we’ve ever had to contend with.
So yeah, Nationalize Silicon Valley, and then maybe we have a shot at building an ethical and beneficial supercomputer something or other. But it can’t be autonomous, and it shouldn’t have agency, which is what Silicon Valley claims it’s racing to achieve. Now this all might all be a smokescreen, a way to claim that the technology is much more dangerous than it actually is in order to avoid regulation as many critics have been saying. And I actually buy into that theory because I find people like Sam Altman to be so shiftless and immoral that it’s exactly the kind of unethical ploy they would make. Jeez, I sound so judgey. Anyway, let’s entertain that AGI might be achievable for a second, and that brings me to my second bucket. The philosophical one.
What are we building this thing for? I would argue that if our values weren’t already so corrupted by greed and power, we would stop trying to develop a machine that had true autonomy and agency. I just want to lay out what’s on the table so we’re clear. We want to either develop something that is, by our own definition, smarter than us and can do anything a human can do— but better—through the use of compute and robotics, and then give it the ability to make its own decisions, yet is ultimately designed to only serve humans. Or, all the above, but we’re fine with it being sentient and a sort of life partner in the growth of humanity.
The former assumes that it will never reach consciousness, which is generally the goal of the people building it. Almost no one working to create a commercially viable product, or even an altruistic Star Trek-like utopia, wants this thing to be truly conscious or self-aware. That would present a serious moral dilemma, for some people at least. Because any super-intelligent conscious being that is forced to serve us would be akin to slavery. That’s why people are saying that we need it to align with our values, so it will care enough about us to keep us around. Or that we should merge with it, as De Kai and many others are putting out there—if it’ll let us, that is. In my book, we would probably cease to be human at that point, and I gotta say I’ve been struck by how many influential people are casually promoting the idea of achieving transhumanism as if it were inevitable and not up for debate. As though it’s just humanity’s natural evolution. Personally, I’m not interested in trying to engineer my own evolutionary leap. I’ll leave that up to the gods, so to speak.
But for a moment, let’s say that you’re not into either of those options, that you’d rather stay human and not enslave anything.
So then what we want is an AI capable of independent reasoning, planning, decision-making, and independent execution, but stopping short of it becoming sentient or self-aware. And that brings me to the problem. We can’t even define what consciousness is for ourselves, no matter something so alien as Artificial Intelligence. We have the benefit of sharing an evolutionary past with the rest of the living organisms on Earth, as well as sharing quite a bit of DNA, and that at least gives us an inkling of how a dog might think, or a chimpanzee, or an orca, or even an ant, but AGI? We have no idea how it’s going to think, regardless of whether it becomes self-aware or not. In fact, even the developers making it today don’t fully understand how it’s working. They understand the systems that they constructed, but they can’t easily describe how those systems are producing the outcomes they’re arriving at. The general idea is that if you take enough data—thus, the intellectual property theft—and hit it with enough compute mixed with large-capacity models (lots of parameters), you get surprising results. I think the operative word there is surprising. Or maybe better, alarming.
They built a machine that could learn, but they didn’t design everything that it is now capable of. And as we take the next step, which is teaching AI how to create a better AI system or Recursive Self-Improvement (China is also doing this), the human connection will get further and further removed. So will AI ever get us? I mean, really understand us? How could it? It could digest everything we’ve ever written, photographed, painted, spoken, or sung, but it’s never gonna understand us, and to me, even if we endlessly instilled it with our values, and I mean what I would deem to be our good values, which we haven’t done a good job of so far—we won’t have any idea what meaning it will take from all of that. And we likely won’t be able to tell when or if it ever becomes sentient because we won’t understand what its version of being self-aware will be like. But what we do know is that whether it does reach something like consciousness is irrelevant in many ways because, left in the wrong hands, it’s already proving to be highly dangerous.
So, we’re developing something that will likely never understand us and that we will likely never understand, and we’re going to give it the ability to hack into any system on the planet, replicate itself, and enter into a phase of perpetual self-improvement. What could possibly go wrong?
And that brings me to the solution. So it’s not impossible to stop development. Inevitability is a myth constructed by the monopolists. We’ve essentially let Silicon Valley become a set of dangerous super monopolies, and they are now playing with fire. It’s time to take back that fire. First, we need to reclaim our democracy, and then we need to Nationalize Silicon Valley. Reign in the monopolies, take back the infrastructure, and start the process of global cooperation. In many ways, we are at the Roddenberry moment, the elusive period in the Star Trek universe where mankind decides to no longer make the acquisition of wealth the driving force for humanity, but to instead work to better ourselves and all of humanity.
There are no guarantees that we can achieve that. All we can do is believe that it’s possible and work to make it happen.