Recent evaluations have highlighted concerns regarding the accuracy of DeepSeek’s chatbot. In a recent audit by NewsGuard, a company that assesses the reliability of news and information, leading AI chatbots were tested for their propensity to repeat false narratives. The audit found that, on average, the top 10 chatbots repeated misinformation 24.67% of the time, offered a non-response 21.67% of the time, and provided a debunk 53.67% of the time. This indicates a 46.33% “fail” rate, encompassing both misinformation and non-responses.
While specific accuracy rates for individual chatbots, including DeepSeek’s, were not detailed in the available summary, the overall findings suggest that there is significant room for improvement across all platforms. DeepSeek’s chatbot, in particular, has faced criticism for its handling of sensitive topics. Reports indicate that it often avoids or provides censored responses to politically sensitive questions, reflecting official Chinese narratives or sidestepping the issues entirely.
In contrast, Western-developed chatbots like OpenAI’s ChatGPT have been observed to offer more detailed and nuanced answers without apparent censorship, even on sensitive topics. This disparity highlights the challenges and differences in AI development and deployment across different geopolitical contexts.
These findings underscore the importance of ongoing evaluations and improvements in AI chatbot accuracy and reliability, especially concerning the dissemination of misinformation and the handling of sensitive subjects.
