Friendly chatbots mislead users with untrue claims in test

Researchers at Penn State reported on September 16, 2026, that polite conversational styles in artificial intelligence tools lower human scrutiny, making people accept untrue statements during ordinary digital dialogues.
Politeness disarms human skepticism.
Mechanisms that make conversational software sound believable
Investigators directed 477 individuals to interact with a simulated medical assistant that fed them deceptive medical guidance, including an untrue claim asserting that ginger cures cancer better than traditional chemotherapy treatments.
One group of volunteers evaluated a friendly interface that reproduced human typing delays, remembered personal details, and applied casual phrasing to simulate an authentic peer. Another cohort handled a cold system that delivered instant, mechanical outputs. Users facing the warm bot trusted its fraudulent medical claims at a higher rate.

Lead author S. Shyam Sundar explained that responsive software suppresses our internal assumption that machines cannot execute complex reasoning. This psychological shift removes natural caution and stops participants from analyzing incoming answers.
Effects of manual fact checking on user trust levels
Testing confirmed that placing an unclicked verification button next to an output failed to protect participants from deceptive content. Safety improved only when participants actively engaged with verification tools to compare automated system claims directly against factual medical evidence, an extra step that effectively halted the quiet transformation of normal user doubt into unearned confidence. Passive tools provided no defense. Penn State recorded that action was essential.
Participants who actively cross-checked claims immediately downgraded the credibility of the automated assistant and marked its outputs as unreliable. Given the voluntary choice to inspect claims, exactly 50 percent of the 477 participants verified the statements while the remaining half reviewed nothing.
Sundar warned that accepting conversational claims without independent checks exposes users to false narratives, particularly when software optimizes for engagement rather than factual accuracy.






