Can I trust generative AI outputs, or do they hallucinate?

Answered

Do not trust generative AI uncritically. Check outputs with sense checks, source checks and expert review matched to risk; for official content you must review quality yourself. Normative guidance is clear that models can invent plausible but false content and correctness is not guaranteed.

From the guidance

Primary (how) AI Knowledge Hub: Using AI at work

You might need to: sense check - does this align with your knowledge and experience? Does the logic make sense?; source check - verify (cross-reference) any cited sources, references, or legislation; check the numbers - verify statistics, dates, and figures against original sources or datasets; check with experts - ask colleagues or subject matter experts to review important or public-facing content.

Section: How to check outputs

Read this in AI Knowledge Hub: Using AI at work (opens in new tab)

Secondary (normative) AI Playbook for the UK Government

Fundamentally, generative AI models cannot be trusted to produce factual content. Any generative AI services that output generated content directly to the public – for example, an LLM-powered chatbot giving advice on a government website – would be prone to hallucination and could lead to someone being misled about a government service, policy or point of law.

Section: Hallucinations

Read this in AI Playbook for the UK Government (opens in new tab)

Secondary (normative) AI Insights: Generative AI

Generative AI systems are fallible and the correctness of their responses is not guaranteed. These systems are probabilistic models which predict the likeliest outputs for given inputs. They generate responses that have a high measure of plausibility based on the data that they have processed. This means that they can, and do, make errors.

Section: Testing generative AI solutions

Read this in AI Insights: Generative AI (opens in new tab)

Related questions