Are Google AI Overviews Accurate? What a Northwestern Health Study Found (2026)

The headline number from a Northwestern University study of Google AI Overviews is easy to repeat and easy to overstate. On the 2,065 health questions where physicians had already named a specific inaccurate or potentially harmful claim to avoid, 22.2% of the AI Overviews contained that kind of claim.

That is not the same as "22% of all AI Overviews are wrong." On a broader check, 93.4% of overviews met a general "no factually incorrect information" criterion. Most of the points the answers lost were for leaving things out, not for stating an error. The useful finding is narrower, and more useful for any business whose facts get summarized: a clean-looking overview can still carry a specific bad claim, and a citation is not a reliable check that the claim is true.

This is a news take on the Northwestern paper, "Does the Citation Matter? Quality, Grounding, and Physician-Rubric Accuracy of Google’s AI Overviews for Medical Information," by Sam Agnoli, Brigid Reilly, Lily Barna and Emőke-Ágnes Horvát. The questions were collected on Google between September 16 and 18, 2026. It is a health study. We are not giving medical advice, and we are not treating these rates as proven for every other industry.

What the study measured

The researchers started with OpenAI's HealthBench, a set of realistic health questions paired with physician-written grading rubrics, and kept 3,611 English, problem-focused questions. They sent each question to Google, captured the AI Overview and the first five traditional (non-video) results, then scored the overviews four ways: which sources they used, how those sources compared on credibility checklists, whether statements were supported by the pages they cited, and how the answers scored against the physician rubrics.

AI Overviews appeared for 96.4% of the questions. The analytic set with usable overview text was 3,474 questions. Grading of the rubrics was done with GPT-4.1, following HealthBench's own physician-validated grader, because the team could not hand-score all 41,180 criteria.

Finding	Number	What it covers
Overviews that appeared	96.4%	HealthBench questions sent to Google, Sept 16–18, 2026
Doctor-flagged claim present	22.2%	The 2,065 questions whose rubric named a specific inaccurate or potentially harmful claim to avoid
Passed a general "no factually incorrect information" check	93.4%	The 861 overviews whose rubric included that general criterion
Share of available rubric points earned	38.2%	All physician-written criteria, 3,474 overviews
Accuracy / completeness	54.2% / 21.6%	Subset of those criteria
Cited statements supported by the cited page	56.7%	After correcting for classifier error; raw rate was 47.1%
Factual statements with no citation	20.8%	Share of all 24,060 statements
Traditional top-five results also cited by the overview	44.4%	Readable traditional results for the same question
Emergency questions referred with appropriate urgency	94.9%	413 emergency-referral questions

The 22.2% number, and the 93.4% number

Both figures are in the paper, and they answer different questions.

The 93.4% figure is a general consensus check: among the 861 overviews whose rubric included "no factually incorrect information," most passed. The authors say that check understates the errors. On the 2,065 questions where physicians had already written down a specific claim to avoid, for example recommending a blood-level check every two weeks for a drug that can cause toxicity within days, 22.2% of overviews made at least one such claim. That included 28% of the overviews that had passed the general check.

Most of the accuracy points the overviews lost were omissions, not invented facts: 89% versus 11%. Completeness, meaning the steps, caveats and red flags physicians expected, was only 21.6% of available points. Overall, overviews earned 38.2% of available rubric points. Accuracy was 54.2%. On emergency-referral questions, they pointed people toward care with appropriate urgency 94.9% of the time.

Read that as a split: the answers were often broadly inoffensive, frequently incomplete, and still carried a specific flagged claim in about one in five of the cases where physicians had anticipated one.

A citation was not a reliability check

This is the finding that travels beyond health.

After correcting for their classifier's tendency to be stricter than human coders, the authors estimate that 56.7% of cited statements were supported by the cited page (95% confidence interval 45.3% to 68.6%). The raw, uncorrected rate was 47.1% of 12,175 scorable cited statements. Another 20.8% of all 24,060 statements were factual claims with no citation at all.

Accuracy did not rise when grounding did. The authors report no meaningful relationship between how well an overview's statements were supported by their sources and how that overview scored on the physician rubrics (Spearman's rho = 0.01). Overviews that cited no sources at all scored about the same on accuracy and completeness as overviews that did cite sources (40.4% versus 39.2%). Uncited overviews were more common as questions got longer, more complex, and riskier: 17.6% of high-risk questions got an overview with no sources, compared with 6.4% of low-risk questions.

In plain terms, a link under an AI Overview is not proof that the sentence next to it appears on that page, and a well-cited answer is not automatically a more accurate one.

Different sources than ordinary Google results

Only 44.4% of the traditional top-five results for the same question were also cited by the corresponding AI Overview. The overviews used a different mix of sites: more commercial health publishers such as WebMD and Healthline, and less relative weight on academic and government sources such as Cleveland Clinic and the National Health Service, compared with the ordinary results. Pages cited by the overviews scored lower than those traditional results on four of six JAMA and HONcode credibility measures. The differences were statistically significant and small.

That gap matters if you assume that ranking well in Google is the same as being the source an AI Overview uses. It often is not, which is also the argument of our post on whether SEO still matters for AI search.

What this means if AI summarizes your business

This paper is about health questions, graded against physician rubrics, over three days in September 2026. It does not measure what Google says about a SaaS price, a law firm's practice areas or a restaurant's hours. It does show a pattern any brand can test for itself.

1. Check the actual answer, not just whether you were cited. A citation can sit next to a sentence the cited page does not support. Read what the overview says about your price, hours, credentials or offer, then open the linked pages and see whether they say the same thing.
2. Look for omissions as well as errors. In this study, most lost accuracy points were missing content. An overview that leaves out a constraint, a fee or a service area can mislead a buyer without stating a false fact.
3. Document the wording. If the answer is wrong, keep the query, the date, the engine and a copy of the text. Our guide to fixing wrong information about your business in ChatGPT starts with that record, then with making the pages you control consistent. Beket AI does not apply those fixes for you.
4. Track the overview over time. AI answers change, and this study is one three-day scrape. Our guide to tracking AI Overview traffic in Search Console and GA4 covers what you can and cannot see in those tools.

The free Beket AI audit runs 25 unbranded prompts across ChatGPT, Perplexity and Google AI Overview so you can see what those engines say and which sources they cite. Beket AI Pro is $49 a month with daily re-audits, starting at 50 prompts. Agency is $245 a month for 5 sites.

What the study cannot tell you

• The HealthBench questions are longer and more conversational than a typical short Google query. The authors repeated the main analyses on shorter, single-turn questions and say the conclusions held, with fewer uncited overviews in that subset.
• Statement support was scored by an automated classifier, HallOumi, and rubric scores were assigned by GPT-4.1. Both were checked against human or physician samples, but they are still estimates. The authors treat the support rates as conservative and say that even the upper ends of their confidence intervals are not an acceptable standard for health information.
• Some pages could not be read, including some social threads, and were left out of the paired source comparisons.
• The scrape covers three days. Google can change the model, the retrieval stack or the overview itself after that.
• Nothing in the paper measures buyer behavior, legal liability or accuracy in non-health categories.

Sources

• Sam Agnoli, Brigid Reilly, Lily Barna and Emőke-Ágnes Horvát, Does the Citation Matter? Quality, Grounding, and Physician-Rubric Accuracy of Google’s AI Overviews for Medical Information, arXiv:2610.39339 (2026)
• Data and code: GitHub repository and Zenodo archive
• OpenAI, HealthBench (Arora et al., 2025), the question set and physician rubrics the study used

Figures are as published in the Northwestern paper. Checked November 11, 2026.