AI Companies Avoid Post-Launch Safety Research: A Surprising Research Bias Revealed by 9,439 Papers

AI companies announce their "safety initiatives" daily. However, a scrutiny of over 9,000 papers reveals a clear bias: They are enthusiastic about research before a model is released to the public, but surprisingly reluctant to conduct research after it's actually being used. This article introduces the research study "Real-World Gaps in AI Governance Research," which makes this subtle but important point.

A Meticulous Analysis from 9,439 Papers

This paper is by Ilan Strauss and others from the AI ​​Disclosures Project (under the Social Science Research Council). Ilan is also a researcher at UCL, and the co-author is Tim O'Reilly, founder of O'Reilly Media.

The analysis is based on a massive corpus of 9,439 generative AI-related papers published between January 2020 and March 2025. From this corpus, papers from five companies (Corporate AI), including Anthropic, OpenAI, and Google DeepMind, and six universities (Academic AI), including CMU, MIT, and Stanford, were extracted. From these, 1,178 papers related to "safety and reliability" were further selected for analysis. The careful handling of co-authored papers spanning multiple institutions, employing a "proportional author allocation method" that allocates the contribution of each institution according to the number of authors, is commendable research design.

The Premise that Corporate Research Has "Overwhelming Influence"

First, as a foundational fact, while the number of papers published by companies is fewer than those published by universities (1,527 vs. 3,578), the number of citations surpasses that of universities (119,845 vs. 78,858). In particular, the number of citations for general generative AI research by Google DeepMind (69,453) exceeds the combined total of the top four universities analyzed. This establishes the premise that the "voice" in AI research is extremely concentrated in a few giant corporations.

Categorical Analysis of Corporate Research Focused on "Pre-Deployment" Stages

The paper then classifies safety research into eight categories (alignment, testing & evaluation, interpretability, ethics & bias, post-deployment risk, multi-agent safety, policy & governance, privacy & security) and compares the emphasis placed by companies and universities.

The results are clear. The influence (number of citations) of corporate research is primarily driven by research conducted before a model is actually released to the market, specifically "alignment" and "test & evaluation". Conversely, there's an interesting reversal: academics have a far greater presence than corporations in "ethics and bias" research, which deals with the behavior of models after they've been commercially deployed.

The authors' observations on this trend are frank: "Much of the research called 'test & evaluation' actually deals with the pre-deployment context. So-called 'in-the-wild' evaluation attempts to predict how a model will behave after it's actually deployed, but it's essentially retrospective." This suggests a structural limitation: reliance on benchmark datasets based on known failures prevents the capture of newly emerging risks.

The Lack of Research in "High-Risk Areas" as Told by Numbers

Particularly striking in the paper is the table showing the lack of research investment in high-risk areas that have led to actual lawsuits, such as healthcare, finance, misinformation, and addiction. In the medical field, there are 53 papers from universities compared to only 9 from companies. Similarly, in papers on misinformation, there are 53 from universities compared to only 8 from companies. This is a significantly larger disparity than the general ratio of academic papers (approximately 2.5 to 1 between universities and companies).

The specific examples cited in the paper are quite stark. It discusses the case of Character.ai, which is being sued over "bots designed to create addiction," and then cites the example of Meta, which subsequently expanded the scope of explicit language permitted in romantic role-playing in its AI chatbots. The paper points out that while corporate system cards mention the risk of persuasion, they often lack concrete safeguards to address it.

Reference to Anthropic's Own Research

What makes this paper particularly interesting is that it specifically examines the research stance of Anthropic itself. The paper points out a disparity in research investment between Anthropic's efforts to conduct interpretability research for risk detection based on the "internal workings of the model" and to validate Claude's values, and the company's own published abuse case reports—specific examples of actual abuses of Claude, such as recruitment fraud, malware development, credential theft, and management of social media bot networks for political influence.

According to the paper, Anthropic's report itself acknowledges the continued risk of future problems, stating that "as agent-based AI systems evolve, these semi-autonomous, organized, and complex abuse systems are expected to continue." Nevertheless, the paper argues that research investment in safeguards for such real-world applications is not commensurate with this risk.

Why does this bias exist?

The authors cite two factors behind this trend. One is simple commercial incentives—the structure makes it unlikely that companies will proactively publish research findings that could negatively impact product market penetration or regulatory treatment. Another factor, the paper points to as an ideological one, is the influence of a philosophy that emphasizes existential risk (x-risk). Under this philosophy, the autonomy inherent in the model itself is considered the primary source of risk, and concern for catastrophic future scenarios tends to take precedence over immediate operational concerns.

Policy Recommendation: Learning from Financial Regulation's "Suspicious Transaction Reporting" System

The latter half of the paper offers concrete recommendations for correcting this information asymmetry. Of particular interest is the framework modeled after the financial industry's "Suspicious Transaction Reporting (SAR)" system. Similar to the "safe harbor" provision that exempts financial institutions from civil liability for good-faith reporting while obligating them to report suspicious transactions to authorities, the paper proposes that AI companies should also have a system in place to disclose operational data (logs, system operation records, etc.) to certified researchers and auditors, exempting them from the risk of liability.

A Note from a Researcher's Perspective

This paper itself has limitations, as acknowledged by the authors. It uses AI models (such as GPT o3-mini) for part of the classification process, and its classification accuracy may not be perfect. Furthermore, it doesn't fully capture the realities of collaborative research between companies and universities (such as the movement of personnel). Also, the paper is based on data from May 2025 and does not reflect subsequent changes in the situation.

Nevertheless, I believe the significance of quantitatively visualizing, with data of this scale, what is actually being researched and what is not being researched behind the scenes of companies' flashy announcements about "safety initiatives" is considerable. This paper reminds us of the importance of developing the habit of being aware of what is not being said when reading company press releases, not just what is stated.