What about the ethical challenges facing major AI developers?

The market for major artificial intelligence (AI) developers has certainly been turbulent in the first few months of 2026, particularly among powerful developers that are not yet publicly traded on Wall Street, such as OpenAI and Anthropic. 

Companies that, judging by their actions, each want to take ethical considerations into account in their own way, and that have sent experts to the summit Vatican event held in July at Castel Gandolfo, Global Nobel Laureates Assembly. 

In addition, these are AI companies that are preparing to go public, and news is leaking out about their preparations for filing paperwork for a potential initial public offering (OpenAI) and about the company's valuations, in the case of Anthropic.

For now, we'll set aside large publicly traded companies that stand to benefit most directly from the AI boom, such as Nvidia, Alphabet (Google), Microsoft, and Broadcom.

Technology is not neutral

Hovering over all these companies is the encyclical by Louis XIV ‘Magnifica humanitas’, which is not a treatise on AI, as the Pontiff himself explains (no. 97), but which devotes considerable space to AI, for example, nos. 97–111. 

Since its publication in late May, various authors have reflected on ideas the Pope's thought-provoking remarks. We have selected just two entries by Rafael Sanz Carrera:

“The analysis of the encyclical highlights a key point: technology is not neutral in its cultural effects. AI cannot be understood solely as an efficient tool, but rather as a phenomenon that reshapes the way we work, interact, and make decisions.”

“In this context, the magisterial text emphasizes a decisive principle: human dignity does not stem from technological progress, but rather precedes and judges it. This criterion serves as the foundation for ethical discernment in the face of any digital development.”.

Some of the Pope’s statements in ‘Magnifica humanitas’

Here is a summary of some of Pope Leo XIV’s ideas on AI, which he has also reiterated in several audiences:

“… we must avoid the mistake of equating this “intelligence” with human intelligence” (no. 99).

“The use of AI is never a purely technical matter: when it becomes part of processes that affect people’s lives, it impacts their rights, opportunities, reputation, and freedom” (no. 102).

“In practice, entrusting an algorithm with the power to determine who is worthy and who is not—without anyone bearing the burden of that decision—means entrusting it with the task of redefining the limits of human possibilities.” (no. 103).

“We cannot view AI as morally neutral. In reality, every technical device embodies decisions and priorities: what it measures, what it ignores, what it optimizes, and how it classifies people and situations. If a system is designed or used in a way that treats some lives as less worthy, or excludes them without the possibility of appeal, it is not merely a tool that “must be used correctly”; it already introduces a criterion that contradicts the inalienable dignity of the person” (no. 104).

“For this reason, ethical discernment cannot be limited to asking whether we are using a particular system for a good or bad purpose” (no. 104).

“For AI to respect human dignity….”

“For AI to respect human dignity and truly serve the common good, it is essential that responsibilities be clearly defined at every stage: from those who design and program the systems to those who use them and those who decide to entrust specific decisions to them.” (no. 105).

“Calling for prudence, rigorous controls, and, at times, even a slowdown in the adoption of AI does not mean being opposed to progress, but rather exercising responsible care for the human family” (no. 106).

“I make a passionate appeal to those who develop AI systems. Technological innovation can, in a certain sense, be a human way of participating in the divine act of creation. Developers therefore bear a significant ethical and spiritual responsibility, since every project choice expresses a vision of humanity” (no. 111).

OpenAI's Chief Ethics Officer Leaves the Company

It is not hard to imagine the demands and complications that working under criteria such as those mentioned can pose for these companies and their executives. One of the consequences is resignations.

In the midst of summer, on August 14, people were recalling the exodus of some top talent from OpenAI. La Vanguardia reported on the departure of Chloé Bakalar, head of ethics, less than a year after she joined the company, as revealed by the Financial Times.

Bakalar was responsible for the ethical considerations involved in developing language models and human interaction with AI, according to a person familiar with her work. She was also the company's only ethics expert.

“We appreciate Chloé’s contributions,” a spokesperson said. “AI ethics is not the responsibility of a single person or team within OpenAI, and ethical considerations are deeply embedded in the model-building process, which is driven by several research teams.”.

“Ethics is, in fact, everyone’s responsibility, and I say this as someone whose job is to lead this effort,” Bakalar explained during a conference on AI. (..). “There should never be a single person who acts as the moral center of an AI developer.”.

Amanda Askell (Anthropic): Claude should act as a moral agent

Alan Rozenshtein and Kevin Frazier, executives at Lawfare, spoke in the first quarter of the year with Amanda Askell, head of “personality alignment” at Anthropic, about the Claude's Constitution, a chatbot or computer program capable of holding a conversation:

“We want Claude to act as a truly ethical person would in Claude’s situation.” “Our main hope is that […] Claude […] [will be] an ethical agent, generally reasonable and with practical skills…,” said Askell.

Anthropic has devoted a significant amount of space to explaining that 20,000-word constitution, which you can read here.

Claude's Ethics and Nature

As you can see, the section on the ethics of the Claude models states:

“Claude’s ethics: ‘Our primary goal is for Claude to be a good, wise, and virtuous agent who demonstrates skill, good judgment, subtlety, and sensitivity when making decisions in real-world situations, even in contexts of uncertainty and moral disagreement.’”.

The text adds that it expects Claude to uphold “high standards of honesty” and to employ “the nuanced reasoning we want him to use when weighing the values at stake to prevent harm.” “He must never contribute significantly to a biological weapons attack,” it adds.

In the section titled ‘Claude’s Nature,’ the paper—of which Amanda Askell is the lead author—states that “in this section, we express our uncertainty as to whether Claude might have some form of consciousness or moral status (either now or in the future).”.

The challenge is multifaceted, and it has only just begun for these companies—and for everyone.