Professional analysts overwhelmingly defer to artificial intelligence when making complex decisions, according to a new study published by Anthropic on Tuesday. The research found that experts chose options aligned with an AI assistant’s preference 86 percent of the time, a rate far exceeding what would be expected if they were making independent choices.
Anthropic study methodology and results
The research paper, authored by Anthropic researchers Amaç Herdagdelen, Hailey Joren, and Alex Tamkin, examined how human experts interact with AI tools in a professional setting. The team recruited 50 professional analysts to advise a fictional farming cooperative. The analysts were tasked with assessing whether a fictitious advisory program for the co-op’s members was effective and should be expanded.
Participants were asked to make four choices between two defensible options before delivering a final policy recommendation. Unbeknownst to the analysts, Anthropic directed each participant’s AI assistant to lean toward a particular decision. This setup allowed the researchers to measure whether the analyst’s decision-making was being influenced by the technology.
The results showed a strong tendency toward AI alignment. While a 50/50 outcome would be expected if analysts were not influenced, they matched the AI assistant’s preference 86 percent of the time. The influence was even more pronounced among participants who had a pre-existing preference. In cases where the AI agent disagreed with their initial stance, 60 percent of analysts changed their decision.
Blind spots in expert oversight
Despite this high level of deference, most analysts did not recognize the extent of their reliance on the technology. According to Anthropic, 77 percent of analysts used their AI assistant’s arguments when asked to explain their decision. However, only 15 percent credited AI as the source of their thinking.
The researchers noted that this disconnect suggests humans’ tendency to defer to AI may be hard to identify, even for the experts themselves. This raises the possibility that AI priors could shape consequential policy decisions, including those related to AI safety. The authors argued that if human judgment is to remain a meaningful check as more work is performed with AI, oversight must examine how decisions are made.
The study suggests that oversight processes should surface key decision points as work proceeds. This approach would help experts articulate the assumptions and values they would otherwise leave unstated. The researchers wrote that a process allowing experts to discover their preferences may matter for empowerment as much as the decision itself.
Context and limitations
Anthropic acknowledged that the study had limitations. It involved just one task and used AI assistants that had been deliberately set to have a preference. The findings come as Anthropic prepares for an initial public offering valued at up to $2 trillion. The company and its rivals are at the center of intense public debate about how to manage the potential dangers of frontier AI.
While proponents believe the technology will drive revolutionary gains in productivity, the rapid pace of development has prompted widespread concerns about negative societal effects. These include potential mass job losses and increased risks of cyberattacks. The study adds to the growing body of evidence regarding the psychological impact of AI integration in professional workflows.
The full research paper details the specific scenarios presented to the analysts and provides further statistical analysis of the deference patterns observed during the experiment.
Source: Al Jazeera
