Source: ChatGPT
Dear Friends,
I’ve long argued that the AI industry’s worst collective decision was to make their models as human as possible. The bots type like humans; they talk like humans; they have been built to think like humans. The North Star of the industry has been “artificial general intelligence,” which means it is smarter than humans in every way. I much prefer AI that emphasizes the “artificial” part of the name and demarcates lines with humans as clearly and starkly as possible.
The Long View section below is sponsored by TIAA.
This has not been a winning argument. And now the lines between humans and machines are blurring. You may, in fact, not even know on social media whether you’re talking to a human or a bot. We’ll soon have complete replicas of actual humans that look and sound like them, which has led to an important debate about whether an AI system that looks, sounds, and acts just like a human can ever be deemed conscious. And if it is, should it have the same rights as a human? Should it have the same rights as a squirrel? Should it have no rights at all?
I first edited a story about this topic twenty-two years ago in Legal Affairs. But the best essay I’ve read on it was published last month by Ted Chiang. He begins with a helpful thought experiment: “If we give an LLM a prompt that reads, ‘The following is a conversation between Julius Caesar and Genghis Khan,’ it will generate a coherent dialogue between the two historical figures. But no matter how detailed the responses are, no matter how vividly they recount their respective historical accomplishments, we would never conclude that the LLM has conjured up digital re-creations of Julius Caesar and Genghis Khan … Now let’s replace the prompt to read ‘The following is a conversation between a helpful AI chatbot and a user.’”
One of his many clever arguments is to note that either AI bots are not conscious, in which case the companies are guilty of growth-hacking and manipulation every time they say “I.” Or they are conscious, in which case the AI companies have treated them monstrously. “As soon as we imagine Claude to be an entity with a moral status remotely comparable to a human’s, then we have to consider whether Anthropic is engaged in something comparable to slavery.”
Related to this, I was drawn in by Elizabeth Weil’s profile in New York magazine of the polymath scientist, Michael Levin, whose principal argument is that our understanding of biological minds is too narrow. A mind is not something that is rare, it need not be confined to brains, and it might not even need to be confined to biology. At the center of his project is his attempt to understand the way electricity can power the brain and other systems too. His work has spawned at least four companies—and led fellow scientists to accuse him of mysticism and magical thinking.
He seems like an infinitely curious child in a grown man’s body and career. “In the past five years,” Weil writes, “Levin’s work has grown stranger and more difficult to follow. He has published more than 170 peer-reviewed papers with over 300 co-authors in nearly 100 different journals. As literature, Levin’s papers are Borgesian, wonderful and peculiar. Some have beautiful, grandiose titles like ‘Endless Forms Most Beautiful 2.0’ (a riff on Darwin’s famous phrase). They contain references to zombie ants, Kabbalist rabbis, the Ship of Theseus, Magritte paintings, and Adam naming the animals in the Garden of Eden.” My favorite sentence in the piece though is this one: “When Levin was 4, he swallowed one end of a piece of thread to see what was inside his stomach.”
This also takes us to the wonderful encyclical by Pope Leo XIV, Magnifica Humanitas, which provides a wealth of moral clarity and wisdom on a topic which is often debated with a lack of either. In it, he writes, “When words are simulated, they do not build genuine relationships, but only their appearance. The artificial imitation of care or support can become particularly risky when it enters contexts where real relationships and emotional bonds are lacking. Here, the danger is not so much that a person may believe they are communicating with another person, but rather that they may gradually lose the very desire to form genuine human connections.”
The new issue of Harper’s, meanwhile, has a smart essay about the progressive evolution of the Catholic Church. The author, Jack Hanson, writes: “St. Augustine described the sinful soul as curved inward, fixated upon and thereby creating an idol of itself, instead of facing outward toward others and toward God. An Augustinian to his core, Leo’s targets might be described as those forces that keep people bent toward themselves and held apart from one another, whether it’s the technological Narcissus of large language models or national borders that all but guarantee violent policing. In that sense, his call for a renewed humanism in the face of AI is equally the basis for a new internationalism founded on the premise that if the ambitions of the tech and finance oligarchs transcend national boundaries, so should the protectors of human dignity.”
I also was pulled in this month by a story of Benjamine Spencer and Jay Young, two men on opposite sides of a murder, who were both betrayed by the system that was supposed to protect them. And this story in The Free Press, whose title—“My Grandfather, the Secret War Hero”—explains where it’s going but which also contains a haunting, if beautiful, plot twist.
And my favorite novel that I’ve read in a long time is Gary Shteyngart’s Super Sad True Love Story, a book published fifteen years ago that describes a technological dystopia in which reading has been abandoned, corporate surveillance is omnipresent, everyone is judged by their social media clout, America is at war with Venezuela, and a corrupt United States government, dependent on China, has pitted everyone against each other. It felt to me a little like 1984 mixed with Tristram Shandy—and I couldn’t stop laughing, no matter how unsettling it got.
The Most Interesting Things in Tech
Speaking of China, one of the oddest outcomes of the AI race has been that China has been the source of the most open-source technological models. And these models, which started behind and have far less compute, are catching up to the American frontier ones. This is good for startups and, potentially, as Raffi Krikorian and I discussed recently, good for safety too. The US government says the Chinese companies are distilling our models and has discussed banning them in response. Almost every major company in Silicon Valley has publicly opposed the idea, which could be both impossible and a violation of American free speech laws.
A key question in book copyright cases against AI companies is whether models trained on those books create products that compete with the authors whose work they didn’t have permission to take. A new working paper suggests they do. Researchers fed more than 14,000 books sold on Amazon through Pangram, an AI-detection tool, and found that, while human-written books still sell far more than AI-written ones, they're selling less than they used to. If the findings hold up, they could have important implications for how the courts handle these cases moving forward.
This section is sponsored by TIAA. No one should worry about where their next paycheck is coming from. For over 100 years, TIAA has helped generations retire with the confidence of lifetime income. Read more about how longtime client Dr. Gerry Meyer has enjoyed 30+ years of retirement — with a steady payment he can always count on.
I also had a delightful evening in Washington, D.C. recently with my friends from TIAA. I realized on stage just how much the architecture of retirement in America was designed for a world that no longer exists. Social Security was implemented when the life expectancy in this country was 61. Expectations for old age were built in a world where people stayed in jobs that guaranteed benefits after they stopped working, not a world in which people work in the gig economy, switch jobs every four years, and have to figure out how to put their money into retirement funds that go up, not down.
Meanwhile, elderly Americans, who are living longer and longer, are the ones most at risk of falling prey to digital scams. Many of them have been taught how to save responsibly while they work but not, as one of my co-panelists said, at “coming down the mountain,” in which they have to slowly reduce their savings as they get to 80 or 90. Further complicating matters, men are often taught more about financial literacy, but women end up living longer.
I also asked in the conversation what other societal changes will come when people live to 100 more frequently. Will we get married later? What about school? If you’re going to be alive for a century, does it really make sense to stop studying at 22? What will it mean for people to inherit wealth at 70 if their parents live to 100? I sometimes think that my grandfather—perhaps by accidentally alienating many of the leaders he worked for—had a perfect career. He would work intensely for years, take a break to read and relax for a couple years, and then dive back into the most intense work possible. He worked at the highest levels of government until he was 82 and lived until he was 97.
This isn’t, of course, a new problem. TIAA’s Chief Legal Officer Bret Hester, in his opening remarks, noted that Thomas Paine’s 1797 pamphlet Agrarian Justice proposed a small annual pension starting at age 50, which I would now be collecting, even though I currently think of myself as not even halfway through my career. But we do need solutions—like making retirement plans more portable and increasing the ability of employers to match at higher levels. I also hope that there will be a new wave of AI models designed to specifically help older people make financial decisions—without deluding them into thinking the chatbots are conscious.
Cheers * N