How I Learned to Fear AI

“It is so difficult. It wants a Pope to settle that!”

—Thomas Hardy, The Mayor of Casterbridge

The proliferating forms and functions of artificial intelligence have loosened many lips. Across the theoretical and the political spectrum, initial reactions ranged from despair to anticipatory obedience to the holy rage of Sam Kriss.¹ Some common themes, though, have begun to emerge. It is surprising and heartening how many people have expressed, in ways small and large, an uneasy sense that things are changing according to rules they do not understand for the benefit of corporations and governments whose motives and whose methods grow more opaque by the day.

There are people—and then there’s the pope. When I heard that Leo XIV’s first encyclical, “Magnifica Humanitas,” was about “Safeguarding the Human Person in the Time of Artificial Intelligence,” my first reaction was dread. Forgive me for flashing back to Galileo: organized religion’s knee-jerk reactions to imaginative novelty and invention have a pretty checkered history. I grew up looking for inspiration in a different corner altogether. My childhood taste for tales of alien adventure has grown over the years into a sense that the most absurd and satirical chroniclers of today’s science and technology were the ones to watch. Ever since Karel Čapek coined the word robot 126 years ago, science fiction has been gloomily prescient on science’s always underplayed risks alongside its highly trumpeted rewards. Back in the Sputnik era, Hannah Arendt cannily focused her sights on “the highly non-respectable literature of science fiction (to which, unfortunately, nobody has yet paid the attention it deserves as a vehicle of mass sentiments and mass desires.)”² I might put it slightly differently: Science fiction has a reliably disinterested kind of satirical detachment, a cold but laughing eye cast down on the foibles of self-serving humans on the planetary surface below. So strongly do I agree with Arendt about the importance of this kind of attention that I even prick up my ears when science fiction writers like Ted Chiang (of Arrival fame) turn to nonfiction. When he sounded a prescient early warning about LLM’s limitations in early 2023 and unleashed another polemic on the delusion that AI models like Anthropic’s Claude are approaching consciousness, he was clearly drawing inspiration from a long tradition of science-fictional skepticism in the face of purported novelty.

However, the more carefully I read Leo’s encyclical, the guiltier I felt for my knee-jerk anti-popery. He cottoned onto the key points in the current AI debate, but in a decidedly un-science-fictional way (the comparisons between satirical science fiction and papal homiletics are probably fodder for another very different sort of Public Books article). You might begin with the fact that “Magnifica Humanitas” pointedly bears a publication date exactly 135 years after Leo XIII’s 1891 “Rerum Novarum,” which famously made the case for what came to be called the “social doctrine of the Church,” rethinking tricky questions of owner-worker relations in light of “workingmen’s unions.”

That intertextual nod back to Leo XIII points to Leo XIV’s main interest. He wants to reconsider capitalism’s power relationships in the light of various new AI technologies that threaten not just human creativity (what most worries me as an English professor) but also the balance of power between technocratic elites and the rest of us: their employees, their consumers, and those unlucky enough to live near their environmentally destructive data centers. Addressing himself to “all the Catholic faithful, to all Christians and to all men and women of goodwill,” Leo proposes a solution to this real-life case of what in science fiction is called the “novum” (the novel technological insight that jolts narrative into gear): “to face the challenges of our time with clarity of thought and responsibility … [by establishing] adequate regulatory tools capable of upholding justice and curbing the distorting effects of technological power.”

Leo somehow manages to achieve a suitably judicious as well as judicial remove, merging the Cassandra power of dire prognostication with an earnest taste for policy tweaks I associate with the best bureaucrats of the New Deal, as if white smoke had been lit for Frances Perkins. Even as someone who prizes satire over homily and genre fiction over theology, I stand in awe of his overall ambition in the first significant missive of his papacy. (Less so its embrace of a “situated anthropocentrism”; more on that later.) Understanding Leo’s ambition, though, entails parsing the key turns of his argument—and grasping its limits.

Love his arguments or hate them, I appreciate the chance to witness this particular pope out there in the agora, making his case.

“Magnifica Humanitas” makes six central claims about AI. First, Leo warns we should be wary of AI’s private (and hence privative, indifferent to the needs of the many) origins.

The main drivers of development are private, often transnational, parties that are endowed with resources and the capacity to intervene that surpass those of many Governments. Technological power thus takes on an unprecedented, predominantly “private” aspect, which makes it even more challenging to discern, govern and direct such power toward the common good. … For this reason it is necessary to begin a shared discernment process for identifying the spiritual and cultural roots of ongoing transformations.

His call to discern ways that the deployment of AI could be governed otherwise, with greater public agency more widely distributed, is not a Luddite refusal of the technology’s existence. Rather it is a reminder that every technical breakthrough can be tuned or attuned to the needs of the profit-seeking few or the less fortunate many; and there have to be incentives—ethical, fiscal and if necessary legal—to keep the attention of developers focused on the right questions.

Second, he stresses the innate preferability of a diversity of views on AI and its uses to a corporate monoculture. This plea for diversity is not shallow but deep, a case for innate human difference as a source of strength, and for regarding the homogenizing profit motive as a threat rather than an equalizing impulse. This has the striking effect of the Church, rather than California-style capitalism, making the case for diversity and inclusion.

We must, then, avoid the “Babel syndrome,” namely the idolatry of profit that sacrifices the weak, a uniformity that neutralizes differences, and the pretense that a single language—even a digital one—can translate everything, including the mystery of the person, into data and performance. The risk of dehumanization—of building a future that excludes God and reduces the other to a means—is an ancient and ever-new temptation that today takes on a technical guise. [Instead, we should strive for the] possibility of building together, of transforming diversity into a resource and of making listening and dialogue the common ground upon which to cultivate justice and fraternity.

By this account it is the Church and not the forces of secularization that most respect and safeguard the greatest variety of human experience. There are familiar Kantian grounds for this insistence on treating others as ends not means. But there is also, behind and before Kant, the medieval notion of multiplicity as valuable in and of itself: I still remember my teen discovery, in Arthur Lovejoy’s work, that an angel may be preferable to a rock yet nonetheless a rock and an angel are preferable to two angels.

This leads Leo to make the case that we need to pay more attention to what benefits AI can produce for the greatest number, and especially for the weakest and most marginalized. Good can only be measured as it affects each of us individually, not via some agglomerating utilitarian calculus.

It is individuals that matter, each and every person, together with their families. Social movements, communal ideologies and grand political proclamations in favor of a population are worthless unless they lead to the flourishing of persons—men and women—with their inalienable rights.

Third, Leo prizes not speed of thought but deliberation. Quoting Plato, he distinguishes crucially between education, which teaches individuals how to make sense of the world, and mere proliferation of information, which rewards only quickness to master and adapt to the algorithm. Rapid solutions to difficult problems are destructive not only because they are wrong but also because of what they can do to those who learn to rely on information when what we needed was delay, time on task, and all the possibilities that are eventually, painstakingly forced to the surface by thinking slowly.

Education … is a long journey requiring patience, and therefore needs time for development and for engagement with reality beyond appearances. This is a fundamental issue because every technology shapes those who use it. Educating people about the use of AI, then, involves teaching them to decide when and for what purpose it ought not to be used. The speed and ease with which answers or summaries can be obtained risk extinguishing the desire to ask questions, which is a process that bears fruit only over time. As Plato wrote, the deepest and most important things are learned only after much time and effort, by engaging in discussion with others, “striking upon” ideas and experiences together like flint until the spark of understanding is kindled within us.

In E. M. Forster’s diatribe against life at the mercy of an all-knowing computer system, “The Machine Stops,” a character used to living in her little hexagonal cell finds herself flying over a magnificent inhuman landscape. “Those mountains give me no ideas!” she cries in vexation, pulling up her shade. Leo has noticed the same phenomenon: the atrophy, the denial of rocky intrusive reality, that we get if we accept the sort of cognitive offloading AI makes all too easy.

Fourth, when faced with new technologies like AI, we need a new set of binding principles of justice. Today’s accelerationists (like the Gilded Age Leo’s namesake Leo XIII warned about in 1891’s “Rerum Novarum,” subtitled “On Capital and Labor”) are always going to look to employ workers as cheaply as they can and site their most destructive physical plants where the fewest empowered protests will arise. Leo is suspicious of the nominal progress of a society that still wants to employ workers as cheaply and dirtily as possible.

 In that context, he wades briefly into the theory wars. Leo makes the case that the fallacious notion of post-humanism improperly deflects attention from the actual humans who bear tangible responsibility for rapacious, profit-seeking choices.

Justice demands that we prevent the emergence of new forms of exclusion and deprivation of freedoms: individuals and peoples hindered or denied access to basic technologies, communities exposed to invasive surveillance, and social groups penalized by opaque algorithms that perpetuate prejudice and discrimination. Some post-humanist currents even go so far as to envision “second-class” human beings, subordinate to the interests of the elites who consider themselves superior. This troubling prospect becomes all the more serious when combined with technological tools that exponentially increase the capacity for control and selection. Even certain forms of structural indebtedness, which keep entire peoples in conditions of dependence, reflect the same mentality, in new forms, that tolerates relationships of subordination akin to slavery.

That final word, “slavery,” proves crucial for Leo’s fifth argument, grounded on an explicit genealogy that links present-day AI practices to the ways that the privilege amassed power in earlier centuries: both the explicit practice of slavery and the more diffuse practices of colonialism that corralled the less powerful into situations where they had no choice but to labor, for a pittance, for the benefit of the rich. As Joseph Conrad put it in Heart of Darkness (1899): “The conquest of the earth, which mostly means the taking it away from those who have a different complexion or slightly flatter noses than ourselves, is not a pretty thing when you look into it too much.”

Leo’s point is that nowadays such exploitation needs a whole new set of alibis. Post-humanism spuriously assigns pseudo-agency to machines (as if the real questions of agency here were between skilled engineers on the one hand and living “AI agents” on the other) and makes exploitative wage slavery, or piecework precarity, seem reasonable. There is a similar case for AI-fueled neoliberalism as neocolonialism in Karen Hao’s well-reported Empire of AI (2025). Ditto in Sarah Wynn-Williams’s Careless People (2025), which documents, among other things, Facebook’s rapacious data harvesting and its studied indifference to the traumatic effects of serving as an underpaid content moderator in the Third World.

The genealogical importance of this link—that yesterday’s slavery and colonial exploitative practices offer a template for tomorrow’s slavery to companies grown rich on AI—spurs Leo to rehash and extend earlier papal apologies about slavery:

It was only in the nineteenth century that a formal, absolute and universal condemnation of slavery [i.e., by the Catholic Church] was clearly articulated, notably under Pope Leo XIII. This development offers a clear example of the Church’s growth in understanding the perennial truths of Revelation that she safeguards … It took eighteen centuries for [“the dignity of every human being’s”] full incompatibility with slavery to be explicitly recognized. This constitutes a wound in Christian memory, one from which we cannot consider ourselves detached.… This is why the memory of past complicity and blindness in the face of the injustice of slavery becomes a call to vigilance. What we have learned must be translated into discernment and responsibility in the present.

As Leo frames it, we should look beyond nominalism to see where the reality of such oppression founded on inequitable resource distribution continues today under other names.

Finally, in a section that has received comparatively little attention, the “Magnifica” indicts AI’s looming use in weaponry.

Sometimes there is talk of “artificial moral agents,” as if machines were able to distinguish between right and wrong with greater consistency than a human being. Yet moral judgment cannot be reduced to calculation, for it involves conscience, personal responsibility and the recognition of the other as a person. Therefore, it is not permissible to entrust lethal or otherwise irreversible decisions to artificial systems. No algorithm can make war morally acceptable. AI does not remove the intrinsic inhumanity of conflict; indeed it can only bring about conflict more quickly and render it more impersonal, lowering the threshold for resorting to violence, transforming defense into threat prediction and thus reducing victims to data.

This point about the necessity of preserving human judgment is plenty strong, but Leo ventures farther:

It is not enough to invoke a generic type of ethics. Concrete criteria for discernment must be established. The first such criterion concerns personal responsibility. … The second criterion pertains to the moral timeframe for making judgments. … The third criterion is the identification and protection of civilians. Any technology that facilitates attacks without seeing the face of human beings lowers the moral threshold of conflict. … These criteria give rise to certain non-negotiable requirements … it is imperative to establish a shared framework—also at the international level—in order to curb the technological arms race and ensure robust protection for civilians and the infrastructures necessary for their survival.

Leo locates these automated decision-making systems inside a much older tradition of “just war / justice in war” debates. This debate goes back to sword versus spear or arrow: If you intend to kill your opponent, you ought to be willing to make that decision face to face. A similar debate arose about the moral hazard of operating killing drones in Afghanistan or Iraq from a complex in Virginia. But Leo makes this also into a question of what we lose when we practice cognitive offloading by turning fatal strikes into “decisions” by a machine we have programmed to think and now to kill for us.

---

Leo’s six principles as regards AI’s present dangers are clear:

1. The private profit-seeking motives of current AI practice are leading it astray;

2. Diversity of viewpoints and practices that maximize individual freedom are far preferable to a capitalist monoculture both ethically and practically;

3. Deliberation is preferable to processing speed, and, as Plato taught us, education wins out over proliferation of information;

4. In these novel circumstances we need a new social doctrine that avoids the empty talk of posthumanism;

5. A viewpoint that specifies how the flaws of the past, especially slavery and colonialism, color the abuses of the present even if their continuation takes different names and forms;

6. Warfare waged by and through AI-guided weapons systems works to erode certain implicit frameworks of shared risk that have discouraged escalation and forced individuals to take responsibility and assume risk when moving against their fellows.

Of course, taken together, these six arguments do not fully convey what is at stake in Leo’s encyclical and what further discussions it may spark down the road. There are even wrinkles to the encyclical that Leo may come to regret. He has, for example, shifted his theological focus substantially away from Pope Francis’s “Laudato Si’” (2015), which praised our common home, the Earth, and the sanctity of all its varied inhabitants, and denounced a “tyrannical anthropocentrism.”³

Leo criticizes the uncomfortable and improbable notion of tech surpassing human intelligence by invoking, against machinery, the divinely sanctioned inescapability of human bodies and by stressing a “situated anthropocentrism” in order to win his battle against the martial and mindless threat he sees AI posing. I worry that in centering humans, however, Leo risks downplaying a long Catholic theological tradition, one that Francis brought vividly back to the fore, that calls on humans to “respect nature” and understand the world as a home to all living beings. (Though this is not the place to trace the links, I also wondered if Leo’s regrettable reference to life’s onset at conception and his condemnation of “induced abortion, killing of the innocent and euthanasia” may also be connected to this wrongheaded anthropocentrism.)

Given that Leo frames the struggle as between the Church’s eye for human suffering and a fallacious post-humanism propounded by AI companies, maybe it is asking too much for Leo to condemn anthropocentrism as completely as “Laudato Si’” does. Still, Francis had the right emphasis, and Leo’s notion of “situated anthropocentrism” risks “un-worlding” our species, whom we ought to consider not just in relationship to their technology but also with an eye to the human-induced climate change of the Anthropocene. As Darwin put it in The Descent of Man and Selection in Relation to Sex (1883), “with all these exalted powers—Man still bears in his bodily frame the indelible stamp of his lowly origin.” That common stamp ought to be most palpable in crisis moments. The fact that one of the leading AI firms is called Anthropic is in that context not so much a reassurance as a warning about where its attention is focused.

The ethical crisis moment of AI, if it has arrived, carries some opportunities. Desperate times can spark desperate countermeasures. Although bomb architect J. Robert Oppenheimer may not actually have quoted the Bhagavad Gita right after the Trinity atomic test—“I am become Death the Destroyer of Worlds”—we like to picture him doing so because it marks the magnitude of the moment. Chris Olah of Anthropic standing alongside the pope at this encyclical’s unveiling was less poetical in declaring that “every frontier AI lab—including Anthropic—operates inside a set of incentives and constraints that can sometimes conflict with doing the right thing.” But it does suggest some willingness among AI execs to ponder their own relationship to the results of their technological prowess.

In that moment, even situated anthropocentrism may not be the theologian’s best friend. If there is one thing we know from science fiction, it is that shocking moments of human hubris and overreach often spawn the greatest, most genuine reactions: Slaughterhouse-Five is about the hubris of Vietnam but also about indifference to civilian casualties that World War II’s weapons of mass destruction encouraged. And (although this aspect of the novel drops out of its film version, Blade Runner) Philip K. Dick’s 1968 Do Androids Dream of Electric Sheep? proposes that in a world of sentient, cognizant AI, the human capacity to care for sheep, for frogs, even for spiders is what keeps hope afloat. Dreaming is the index of excess, the sign that not everything in human ethics and the tricky problem of judgment can be solved in the daylight hours. That humans are far from the only animals who dream ought therefore to be a warning, but also a comfort.

I agree with Cal Newport (“Dear AI Companies: The Doom Trolling Needs to Stop!”): Silicon Valley’s current hand-wringing is hypocritical and manifestly self-serving. But that does not preclude all of us, even Anthropic execs, from grasping that the current situation does not call for moderation and safeguards on the part of any one AI company but for a radical rethinking of what these tools will be used for and who will profit from them. Bernie Sanders’s call for a sovereign wealth fund made up of 50 percent of the value of the companies is one good beginning.⁴

In a different sort of essay, this would be an excellent time to think about the ways speculative genres assemble a response not just to science’s cutting edge but also the religious response to scientific “novums.” H. G. Wells’s account, in The Food of the Gods and How It Came to Earth (1916), of the genre’s mission to chronicle “the greatness and the littleness of these scientific men” is memorable because science fiction’s laughter undermines surety, punctures hypocritical piety, and offers in its place Menippean satire, which reveals the ways various mental pictures hold us captive.

Leo, although he surprised me by quoting Tolkien’s Lord of the Rings in the “Magnifica,”⁵ would likely have little patience for science fiction’s satirical antics in this particular arena. Great the “Magnifica” may be; funny it is not. And perhaps that is the point. There are as many cautionary responses to AI right now as there are serious people mulling the topic. The best writers—from Leo to Ted Chiang—proceed from wildly divergent angles. Science fiction’s Menippean satires offer one way to see around AI; the pope offers another.

Perhaps the most encouraging aspect of the discussions that have raged over the last year is how many other ports seemed to have opened. Politicians have started working across the aisle to propose legislative frameworks within which AI can function. There is also the recent Leiden Declaration from mathematicians interested in the role AI has to play in their field. I appreciate that declaration’s invocation of the UNESCO Recommendation on Open Science and especially its emphasis on underlying responsibility: “Affirm the humanity of authorship: Credit and responsibility continue to belong to humans within the mathematical community and should not be given to automated systems. Artificial intelligence may obscure, but does not replace, the collective human labor behind a result.”

Even if a thousand flowers bloom, though, it would still be a grave error to underestimate “Magnifica” and the mode in which it is written. Any pope has a more imperative register at his disposal: He might (as Ross Douthat wants) declare things sinful in the hopes of corralling Catholics. But such a move—typical of the blinkered and bullying limitation of right-wing American religious thought these days—does little to persuade atheists and may not provide much ammunition even among the believers.

This encyclical lacks the force of a papal bull and carries no claim of papal infallibility. Instead, Leo made his case with words and words alone. The pope’s claim is that the legacy of slavery and the legacy of colonialism are discernible beneath what passes as simple profit-seeking by today’s AI behemoths. If, like Leo, we consider the case slowly, without offloading our cognitive tasks, we too may come to see the colonial genealogy of those data centers and their water wasting, those high-tech practices of data mining and low-waged employment to tune the data.

Love his arguments or hate them, I appreciate the chance to witness this particular pope out there in the agora, making his case. As Francis reminded us when he condemned anthropocentrism, the stamp of our lowly origin is on us everywhere, and there’s no sort of post-humanism that can rescue us from the responsibility to think independently and take responsibility for our actions, to think slowly and for the long term. If science fiction constantly reminds us it is possible to think like a Martian, how nice that we have a pope who wants to remind us what it means to think like a human being.