The Pew Research Center set out to see whether an artificial intelligence could mimic the responses of real participants in its high-quality American Trends Panel. The question was simple: could a computer model, fed with demographic details and past answers, produce survey results that match those gathered from actual people?
To answer this, researchers created a set of digital twins – virtual personas that mirrored real panelists. Each twin received the same demographic profile, self-reported background, and answers to a 2025 political typology questionnaire. Using Anthropic’s Claude Opus 4.6 the twins were then presented with three surveys from early 2026 (waves 185, 190 and 192) and asked to answer each question in the exact order and format the human respondents did.
How the experiment was built
The study focused only on ATP members who had completed the political typology survey, ensuring a one-to-one comparison between synthetic and human data. The AI received each question with identical wording, instructions, and programming that the human participants saw. All synthetic results reported below come from Claude Opus 4.6 run on the “low reasoning” setting with extended profile information and an expert-reflection step. The timing of the original surveys was Jan. 20-26 2026, with the synthetic replication occurring Mar. 9-12, Apr. 7-10 and Apr. 27-May 1, 2026.
Where the AI fell short
Across more than 120 questions, the AI’s answers deviated from real-world responses by an average of 12 percentage points – the same error margin seen when looking at all three waves combined. The discrepancy widened on issues that had evolved after the model’s knowledge cutoff, revealing a systematic lag in capturing fresh public sentiment.
Presidential approval mis-measured
When asked about former President Donald Trump’s job performance, the synthetic panel reported a static 46 % approval in both January and. Human respondents, however, showed a decline from 37 % to 34 % in the same period, with the highest real-world figure (47 %) recorded back in February 2025. The AI’s error was especially pronounced among Republicans, over-stating approval by up to 19 points, while its estimates for Democrats were comparatively close.
Timely issues tripped the model
Two recent topics highlighted the AI’s weakness. First, on whether immigration officers should wear face coverings, the synthetic sample said only about 35 % of Republicans found it acceptable, half the true share (≈67 %). Second, regarding U.S. military strikes against Iran, the AI inflated Republican support by 10-19 points across related questions, painting an almost unanimous backing that did not exist in the human data.
Data-center awareness and economic worries
Public knowledge of data centers was another blind spot. While 25 % of real respondents said they had heard a lot, the AI claimed 94 % had heard “a little.” Moreover, the synthetic poll dramatically overstated the belief that data centers are bad for home energy costs, the environment, and nearby quality of life – figures that exceeded 95 % in the AI sample but were far lower among actual adults.
Implications for future polling
These findings suggest that the allure of instant, cost-effective synthetic polls is outweighed by their inability to track rapid opinion shifts, especially after events that occur beyond the AI’s training window. Pew Research Center reiterated its commitment to gathering opinions directly from people and has no plans to replace human respondents with AI models. While AI can aid in exploratory analysis, reliance on it for official public-opinion measurement would risk misinforming policymakers, journalists, and the public.
