The motto of the Department of Health and Human Services (HHS) is “Improving the health, safety, and well-being of America.” Many of the most recognizable government health organizations (e.g. Food and Drug Administration, National Institutes of Health, Centers for Disease Control and Prevention, and many others) reside within this Cabinet-level department. Priorities and policy decisions among the organizations within HHS are critically important to the health, safety, and well-being of Americans. These priorities and policy decisions should be based on careful analysis of available data, with consideration of the risks, benefits, and costs of competing alternatives. The ability to clearly communicate this information is critical to the public understanding and acceptance of these activities, as well as the credibility of the Department.
HHS has issued a series policy explanations justified with problematic sources, including citation of non-existent references. Even when references are correctly cited, the contents have repeatedly been misrepresented. There has been a consistent lack of balance. For instance high-quality studies most accurately representing vaccine safety and efficacy have been disregarded, while low quality studies exaggerating risks have been highlighted.
HHS FAQ to Congress
A few weeks ago I wrote a SBM article discussing the sloppy scholarship in a communication from HHS to Congress. The FAQ from HHS was meant to explain changes in vaccine recommendations, namely, the withdrawl of recommendations for COVID vaccination in healthy children and pregnant women. The FAQ was a mess, with prioritization of low-quality studies that grossly exaggerated risks, while ignoring much higher-quality safety-affirming studies. There was outright misrepresentation of the content of multiple citations. One study was cited as documenting an increased risk of placental blood clotting in pregnant women, but the cited study contained nothing relevant to placental blood clotting. One cited study was in pre-publication and had not been peer reviewed. The journal explicitly stated that the article “…should not be used to guide clinical practice,” yet this article was one of a handful used by HHS to justify a policy decision.
This sloppy document would be laughable if it were not from the most consequential US health organization to the Congress of the United States. If I were supervising a trainee on a project, and if that trainee produced a document of this sort, I would seriously consider removing them from the project. We should expect better from the leadership of our most critical governmental organizations.
This was not the first or last time HHS exhibited sloppy scholarship.
The MAHA Report
On Feb 13, 2025 President Trump issued Executive Order 14212. The report instructed the Make America Health Again (MAHA) Commission to deliver a Make Our Children Healthy Again Assessment. The report was to focus on childhood illness in the US, specifying certain subjects for inquiry, and to deliver the report within 100 days.
The MAHA Commission, chaired by RFK Jr. delivered the report (often referred to as the “MAHA Report”) on May 22, 2025, It was a 78-page report with over 500 references. News outlets quickly discovered troubling errors and irregularities in the report. There were 7 references to non-existent papers. Thirty-seven of the references were entered multiple times. There were numerous misrepresentations of the results of citations. Authors of cited papers note that their papers were inaccurately represented. Some of the references had the strange text string “oaicite” in the URL’s.
In general, the MAHA reaches conclusions consonant with RFK Jr.’s long espoused beliefs and was a dream-come-true for health influencers. The libertarian think-tank CATO institute posted a critique entitled: Premade Conclusions, Post-Hoc Data: The Problem with the MAHA Report. which concludes thusly:
Good public health policy demands humility, precision, and honesty—especially when the stakes are high. The MAHA Report offers none of these. If we want to build a healthier future for our children, we must resist the temptation of prepackaged conclusions and demand evidence-driven thinking, not ideology in disguise. When the government controls both the research agenda and the health advice, it risks turning scientific inquiry into propaganda and personal health into political theater.
How could this happen?
In a document the size of the MAHA Report, it would not be surprising to find a few errors. But there are major errors: non-existent references, incorrect authors and journals, inaccurate reporting of the findings of research studies. What could account for all of these irregularities and inaccuracies? The presence of the “oaicite” string in some of the reference provides a smoking gun. This a fingerprint of use of Artificial Intelligence (AI) in creation of the report”: “oaicite” is a placeholder for a link to a reference to be added later. If for some reason the reference is not added “oaicite” will persist in the output. This text string is meaningless and confusing to the reader but is an artifact of AI output.
Use of AI as a tool is not surprising, or particularly concerning. The presence of errors such as phantom references, duplicate references, and the placeholder “oaicite” are exactly the types of artifacts known to be produced by large language models. The frequency, severity, and obviousness of many of the errors, however, suggest insufficient human supervision and review prior to dissemination of the MAHA Report.
If you prompt an AI large language model to reach a particular conclusion, it will generally comply. I would be very curious to learn what prompts the AI was given in generation of the MAHA report.
An oopsie from the first meeting of RFK Jr.’s Advisory Committee on Immunization Practices.
The Advisory Committee on Immunization Practices (ACIP) is a committee of up to 19 experts convened periodically to advise the CDC with vaccine recommendations and schedules. Members of the committee are customarily filled and vacated on a rotating basis. RFK Jr, in his wisdom as HHS Secretary dismissed the entire membership of the ACIP, and reconstituted it with a streamlined panel of eight handpicked members, including several members with strong anti-vaccine pedigrees. Dr. Gorski discussed the new ACIP here and here.
Lyn Redwood, former leader of Children’s Health Defense, an anti-vaccine group founded by RFK Jr. was scheduled to deliver a presentation at the first meeting of the newly reconstituted ACIP. Ms Redwood is not a member of the ACIP, but holds an ambiguous position in HHS. Her presentation was purported to be a review of the preservative thimerosal in vaccines. Redwood has been a long-time supporter of a discredited link between thimerosal in vaccines and autism. Her report was posted on the CDC website in advance of her presentation. Eagle-eyed observers quickly discovered a fictitious reference. According to Reuters:
It makes reference to a study called “Low-level neonatal thimerosal exposure: Long-term consequences in the brain,” published in the journal Neurotoxicology in 2008, and co-authored by UC Davis Professor Emeritus Robert Berman.
But according to Berman, “it’s not making reference to a study I published or carried out.”
Berman said he co-authored a similarly named study in a different journal – Toxicological Sciences – that came to different conclusions than those suggested by Redwood.
Just to clarify: Ms Redwood supported her conclusions with a fictitious study. If you charitably believe that it was a simple mistake in formatting the reference, one would reach an even more incriminating conclusion: she misrepresented the study she intended to cite.
Coincident with the ACIP meeting, the CDC posted a summary entitled “Thimerosal-containing vaccines and neurodevelopmental outcomes. Review of the evidence.” This was a thorough review of the literature. This report included studies supporting the safety of thimerosal in vaccines, as well as studies suggesting elevated risk. Each study was accurately summarized. The authors of the report pointed out consistent “methodological limitations” in the studies suggesting excess risk. The overwhelming evidence supported the safety of thimerosal, and lack of credible evidence for a causal relationship between thimerosal and autism spectrum disorders. This report was quickly removed from the CDC website but is preserved here.
The creation of this CDC summary demonstrates two things:
- There are professionals within HHS capable of critically and fairly reviewing the literature on a subject.
- The current leadership of HHS is not interested in a critical and fair evaluation of the literature. The report promptly disappeared from the CDC website.
HHS standards vs 2025 HHS practice
Scholarly communication is an art and a science. One of the obligations of scholarly communication is the need to “show your work.” If you make an assertion based on existing science it should be supported. The references should be representative of the best data on the subject and should be accurately portrayed in the document. One of the cardinal rules I learned as a fledgling academic communicator was this: Never cite an article you have not read.
How should we judge the scholarship and scientific communication practices of the 2025 HHS? There are no communication police but HHS does have some internal documents explaining principals and practice standards for scientific communications.
From The Office of the Assistant Secretary for Planning and Evaluation (APSE) comes this document entitled: HHS Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated to the Public.
Within the document is this directive about quality standards:
Quality is to be ensured and established at levels appropriate to the nature and timeliness of the information to be disseminated, and specific standards may be adopted that are appropriate to the various categories of information that is disseminated. Agencies are to develop a process for reviewing the quality of information before it is disseminated. Further, information quality is to be treated as an integral step in every aspect of the information development process.
How rigorously are quality standards being followed when documents contain fictitious references and inaccurate representation of cited studies?
Another document was issued from the Office of Research Integrity entitled: “Avoiding Plagiarism, Self-plagiarism, and Other Questionable Writing Practices: A Guide to Ethical Writing.” The Office of Research Integrity (ORI) resides within HHS. This is a 70 page treatise on responsible, ethical communication in the scientific arena. It deals with plagiarism, accurate citation of the literature, conflicts of interest, etc. This document is written primarily for researchers engaged in writing grants and scientific reports. One would hope that similarly high standards would apply to written documents used to inform decisions about HHS priorities and policies relevant to the health of US citizens.
Here are a few quotes.
Obviously, literature cited in support of our hypotheses must be grounded in sound arguments, tight research methodologies, and flawless data. Citing references known to be methodologically or logically deficient in support of our work is ethically problematic, particularly if we fail to mention these shortcomings. Likewise, if in our search for relevant literature we become aware of important relevant evidence that runs contrary to our data or point of view, we have an ethical obligation to cite such evidence, either in the introduction or the discussion section of our paper. We must not do this dismissively, but in an unbiased manner.
References play a crucial role in scholarly and scientific writing for they allow the reader to explore in more detail a given line of thinking or evidence. For these reasons, it is important that authors strive for accuracy when listing references in manuscripts
…some authors to provide what others view as a biased review of the relevant literature. That is, in placing their data or theory in the context of existing relevant work, authors sometimes cite only references that are favorable to their position. However, consistent with the basic tenets of ethical writing and scientific objectivity, we have a responsibility to cite all relevant material, even work that may contradict our own position. Failure to do so compromises our professional obligation to remain unbiased and is antithetical to the primary mission of a scientist’s search for truth.
Conclusion
The White House released an Executive Order called Restoring Gold Standard Science. In it, the terms “transparent” and “transparency” are used repeatedly as features of Gold Standard Science.
In this article I have presented recent communication practices of HHS and found them lacking. Unfortunately, the communication practices are reflective of a more serious problem: a lack of rigor and lack of seriousness in parsing the best evidence for the purpose of setting HHS priorities and policies. HHS has fallen short in transparency and Gold Standard Science.
Review and compliance with the guidance from Office of the Assistant Secretary for Planning and Evaluation and Office of Research Integrity would be good steps in the right direction.
