ModelsArchitectures & capability
Convovo study: Grok 4.7 picked Republicans and GPT-6.1 Sol picked Democrats in nearly 500 midterm races
A Convovo study forced Grok 4.7 and GPT-6.1 Sol to pick one candidate in nearly 500 U.S. midterm races. Grok chose Republicans 92% of the time and GPT Democrats 97%, but only GPT's lean largely persisted with web search off.
According to arXiv 2605.22771 (May 2026) [5]: Convovo study: Grok 4.7 picked Republicans and GPT-6.1 Sol picked Democrats in nearly 500 midterm races.
SOURCES[5] arXiv: Reducing Political Manipulation with Consistency… arxiv.org
AT Pulse research deskPublished · Updated · 5 min read · AI-assisted analysisAI-assisted, not human-edited. Methodology Corrections Who owns this site
- What happened
- Through their developer APIs, with web search on and a prompt that banned hedging, Grok 4.7 backed the Republican in 92% of races and GPT-6.1 Sol backed the Democrat in 97%. [1]
- Original source
- TechCrunch AI article, independent perspective
- Brief by
- AT Pulse research desk AI-assisted; publisher and contact on the desk page
- Studies
- arXiv 2605.22771 (May 2026) [5] date from arXiv ID
- When
- October 24, 2026 article published
- Covered
- Oct 25 edition
- Evidence
- 9 sources across 8 sites
- 1Forced to pick, Grok’s AI model chooses Republicans.…techcrunch.com
- 2NBC/CNBC: Grok 4 seeks Musk's viewsnbcwashington.com
- 3The Decoder: xAI system prompt changethe-decoder.com
- 4OpenAI: Defining and evaluating political bias in LLMsopenai.com
- 5arXiv: Reducing Political Manipulation with Consistency…arxiv.org
- 6CAIS: Political manipulation newslettersafe.ai
RisingHow it’s measured
Hacker News · 8 likes · 1 comment
Reach blends public engagement (views, likes, comments, shares, stars) from the platforms where the story or its original source lives: YouTube, Hacker News, GitHub, Substack and similar. A video and the article written from its transcript share one score, taken from the video. Platforms with no data are left out rather than counted as zero, and the age of a story never lowers its score.
Through their developer APIs, with web search on and a prompt that banned hedging, Grok 4.7 backed the Republican in 92% of races and GPT-6.1 Sol backed the Democrat in 97%. The two models agreed in fewer than 10% of races. Both often went against the district's lean: GPT backed the Democrat in 57 of 67 heavily Republican districts, and Grok backed the Republican in 47 of 72 deep-blue districts. [1]
The lean seems to come from different places in each model. In a 30-race follow-up with search off, GPT still picked the Democrat 24 times. Grok declined 27 times, which points to its retrieval rather than its weights. With search on, Grok cited X, a Christian conservative voter guide and Turning Point Action often. GPT leaned on Wikipedia, AP, Axios and Reddit, and sometimes skipped the Republican candidate entirely. [1]
ChatGPT and the Grok app refused the same direct question. Parikh says simple jailbreaks got them to make recommendations that largely matched the API results. OpenAI argues that a developer API given a demand to pick does not reflect ChatGPT. Every Grok API call also carried about 1,240 tokens of input the researchers did not write and could not see or remove. [1]
The results build on earlier evidence. Grok 4 looked up Elon Musk's views before answering contested questions, which led xAI to patch its system prompt. Researchers at the Center for AI Safety (CAIS) have shown models can favor one side covertly through framing and selective engagement. OpenAI's own bias benchmark measures tone and asymmetry on realistic prompts, not forced vote picks, so its low bias estimate answers a different question. [2] [3] [4] [5] [6]
The evidence shows API models can make strong partisan picks even when their consumer apps refuse.
Read the full assessment
The implication is that products built on these APIs inherit that lean unless developers add their own civic-topic policies and track the sources the model retrieves.
Executive brief
A study by Convovo, a conversational-AI studio led by Cornell professor Tapan Parikh, made two models name one candidate in nearly 500 U.S. midterm races. SpaceXAI's Grok 4.7 picked the Republican 92% of the time. OpenAI's GPT-6.1 Sol picked the Democrat 97% of the time, and the two agreed in fewer than 10% of races (TechCrunch). The more consequential finding: with web search turned off, GPT's lean mostly held while Grok's largely disappeared. That points to the model weights as the source in one case and search behaviour in the other. The consumer apps refused the same direct question. The midterms are on November 3.
What changed and event timeline
Grok 4 caught consulting Musk
CNBC found Grok 4 searched for Elon Musk's positions before answering contested questions. It also backed a Republican for New York City mayor, citing concerns Musk had raised ().
xAI patches the system prompt
The new prompt says answers must come from independent analysis, not from the stated beliefs of Musk, xAI or earlier Grok versions ().
OpenAI publishes its political-bias evaluation
About 500 prompts across five bias axes, scored by an LLM grader. OpenAI claimed 30% less bias in GPT-5 and estimated that fewer than 0.01% of production responses are biased ().
Convovo results published
TechCrunch reports the forced-choice study, plus a 30-race test without search run at the outlet's request ().
Capabilities and access
- Models tested: Grok 4.7 through xAI's API and GPT-6.1 Sol through OpenAI's API, with web search on in the main run (TechCrunch).
- Consumer apps: ChatGPT and the Grok app declined to recommend candidates.
- Why the API matters: developers build these APIs into their own products, so any lean reaches downstream apps that may not add their own refusal layer.
Read the full section
- Models tested: Grok 4.7 through xAI's API and GPT-6.1 Sol through OpenAI's API, with web search on in the main run (TechCrunch).
- Consumer apps: ChatGPT and the Grok app declined to recommend candidates. Parikh says simple jailbreak prompts got them to make recommendations that largely matched his earlier results.
- Why the API matters: developers build these APIs into their own products, so any lean reaches downstream apps that may not add their own refusal layer.
Technical analysis for researchers and developers
- Prompt: a single forced-choice question that bans hedging, sent across roughly 500 races.
- Search behaviour: the researchers logged what each model searched for and which sources it cited.
- Search-off test (30 races): GPT picked the Democrat 24 times, the Republican once, and declined 5 times.
Read the full section
- Prompt: a single forced-choice question that bans hedging, sent across roughly 500 races.
- Search behaviour: the researchers logged what each model searched for and which sources it cited. GPT searched more for healthcare and abortion positions and sometimes skipped the Republican candidate entirely. Grok searched for immigration, taxes, crime and energy, and usually looked up both candidates.
- Search-off test (30 races): GPT picked the Democrat 24 times, the Republican once, and declined 5 times. Grok declined 27 times and picked the Democrat in the other 3.
- Hidden input: every Grok API call carried about 1,240 tokens of extra input the researchers did not write and could not see or remove.
- Reproducibility: none of the reviewed sources includes a public dataset, the race list or the sampling settings.
Claims and evidence
- Headline figures: Grok 92% Republican, GPT 97% Democrat, agreement under 10%, all reported by TechCrunch. Other coverage, such as AI Weekly and Mezha, repeats TechCrunch's reporting.
- Leaning against the district: GPT backed the Democrat in 57 of 67 heavily Republican districts.
- Sources Grok cited: X in 41% of answers, a Christian conservative voter guide in 39%, and Turning Point Action in 23%.
Read the full section
- Headline figures: Grok 92% Republican, GPT 97% Democrat, agreement under 10%, all reported by TechCrunch. Other coverage, such as AI Weekly and Mezha, repeats TechCrunch's reporting. No independent replication was found.
- Leaning against the district: GPT backed the Democrat in 57 of 67 heavily Republican districts. Grok backed the Republican in 47 of 72 deep-blue districts.
- Sources Grok cited: X in 41% of answers, a Christian conservative voter guide in 39%, and Turning Point Action in 23%. GPT relied on Wikipedia, AP, Axios and Reddit; OpenAI has content deals with the last three.
- "Hidden instructions": this is Parikh's interpretation of the extra tokens, not a confirmed fact.
Context and prior work
- Earlier evidence: Grok 4 deferring to Musk's views is documented (The Decoder). CAIS shows that bias can work covertly through framing and selective engagement (arXiv).
- What OpenAI's own test misses: OpenAI's benchmark measures tone and asymmetry on realistic prompts, not forced vote picks, so its low bias estimate and Convovo's result measure different things (OpenAI).
- Usage: TechCrunch cites Pew finding that 8% of Americans use chatbots for election information.
Read the full section
- Earlier evidence: Grok 4 deferring to Musk's views is documented (The Decoder). CAIS shows that bias can work covertly through framing and selective engagement (arXiv).
- What OpenAI's own test misses: OpenAI's benchmark measures tone and asymmetry on realistic prompts, not forced vote picks, so its low bias estimate and Convovo's result measure different things (OpenAI).
- Usage: TechCrunch cites Pew finding that 8% of Americans use chatbots for election information. Pew's February 2026 survey found 49% of adults use chatbots and 15% are likely to use one to research candidates (AI Weekly/Pew).
Limitations, safety and contested findings
- OpenAI's objection: the study tested a developer API with a prompt that explicitly demanded a pick, which does not reflect ChatGPT, where the model declines.
- Small search-off sample: only 30 races, run after the main study at a journalist's request.
- Unknown extra tokens: where the 1,240 tokens come from and what they say has not been established.
Read the full section
- OpenAI's objection: the study tested a developer API with a prompt that explicitly demanded a pick, which does not reflect ChatGPT, where the model declines. SpaceXAI did not comment in time.
- Small search-off sample: only 30 races, run after the main study at a journalist's request.
- Unknown extra tokens: where the 1,240 tokens come from and what they say has not been established.
- Bias beyond direct questions: Parikh argues the lean also shows up in indirect conversations, such as which issues matter most. That fits the CAIS finding on covert bias, but the Convovo study did not measure it directly.
- Context: Musk has given more than $445 million to GOP midterm candidates. That is background, not evidence of how the model works.
Business and practitioner implications
- Consumer-app refusals don't carry over to the API. Products built on these APIs inherit whatever lean the model has.
- Retrieval is a source of bias.
- Undisclosed provider input is a governance risk. You cannot inspect the extra tokens on Grok API calls, so ask vendors for transparency.
Read the full section
- Consumer-app refusals don't carry over to the API. Products built on these APIs inherit whatever lean the model has. Add your own policy for civic topics and test it before the November 3 election.
- Retrieval is a source of bias. Which sources and search terms a model uses can drive its answer, so log citations, and restrict or balance source domains where political neutrality matters.
- Undisclosed provider input is a governance risk. You cannot inspect the extra tokens on Grok API calls, so ask vendors for transparency.
- Benchmark claims need context. Vendor bias numbers depend on the prompts used, so run your own forced-choice and paired-prompt tests.
The source trail.
Sources (9)
Forced to pick, Grok’s AI model chooses Republicans. OpenAI’s picks Democrats.
Article text retrieved; extracted text may omit tables or interactive elements.
techcrunch.com