Anthropic co-founder Chris Olah has warned that artificial intelligence development should not be guided only by technology companies, arguing that governments, religious leaders, and civil society need a stronger role in shaping how powerful AI systems affect workers and society.
Olah made the remarks on Monday at the Vatican, where he spoke during the presentation of Pope Leo XIV’s first encyclical on artificial intelligence. Olah’s message is a warning that AI development left solely to tech firms risks displacing human labor and needs oversight from governments, religious institutions, and civil society.
Olah says AI cannot be left to tech companies alone
Olah’s central argument is that the future of AI should not be decided only by the companies building frontier models.
Reuters reported that Olah said the development of artificial intelligence cannot be left solely to technology companies, and that broader oversight is needed from religious leaders, governments, and civil society.
Global Banking & Finance Review also highlighted the same point, saying Olah called for inclusive oversight of AI development beyond Big Tech influence.
The statement is significant because Anthropic is one of the most prominent AI companies in the world and is widely known for building Claude, a major rival to models from OpenAI, Google, Meta, and other AI labs. Olah’s remarks suggest that even people inside leading AI firms are concerned about leaving AI governance only to market incentives and corporate priorities.
Labor displacement is a major concern
Olah also raised concerns about the effect of AI on jobs.
Olah said there is a real possibility that AI could displace human labor at very large scale. Global Banking & Finance Review repeated that warning and said Olah connected large-scale labor displacement with the need for stronger societal support.
Olah said that if such displacement happens, supporting affected workers would become a moral imperative of historic proportions.
That framing matters because AI policy is often discussed through productivity, innovation, national competition, and economic growth. Olah’s comments place the focus on the people who may be disrupted if AI systems replace or reshape large categories of work.
Frontier AI labs face conflicting pressures
Olah also acknowledged that AI companies operate under pressures that may not always align with the public interest.
Companies like Anthropic face strong commercial, geopolitical and personal pressures that can conflict with broader societal interests. Olah warned frontier AI labs face incentives and constraints that may clash with doing the right thing.
Olah said every frontier AI lab operates inside incentives and constraints that can sometimes conflict with responsible decision-making, even when researchers are well-intentioned. This is why Olah sees outside scrutiny as essential.
This is one of the strongest points in the speech. The concern is not only that companies may act badly. It is that even responsible companies may be shaped by competition, investor expectations, geopolitical pressure, and the race to release more capable systems.
Vatican setting adds moral weight to AI debate
The setting also made the message more symbolic.
Olah spoke at the Vatican ahead of Pope Leo XIV’s presentation of “Magnifica humanitas,” his first encyclical focused on the rise of artificial intelligence.
The Vatican setting places AI oversight in a broader ethical conversation, not just a technical or business one. By speaking alongside the pope, Olah helped frame AI as a social and moral issue involving human dignity, labor, public accountability, and institutional responsibility.
AI governance debate moves beyond Silicon Valley
Olah’s remarks come as governments and regulators around the world continue to debate how to manage fast-moving AI systems.
For Big Tech, the message is clear: technical leadership does not automatically equal moral authority. For governments and civil society, the challenge is to create oversight that protects workers and the public without blocking useful innovation.
The larger issue is that AI is becoming too powerful, too economically important, and too socially disruptive to be governed only from inside the companies building it.
Olah’s warning suggests that the next phase of AI governance will depend not only on better models, but on stronger institutions outside Silicon Valley.
