Chatbots
Why your bot hallucinates, and what to do
The most common cause of hallucinations in enterprise bots is not the model — it is the retrieval.
Published by CloudFixies · 04 November 2025
Retrieval, not reasoning
If the top-5 retrieved documents don’t contain the answer, the model will invent one. Fix retrieval first: reranker, embeddings tuned for your domain, chunk size that fits your content.
The 30-minute test
Pick 30 real questions. Log the retrieved chunks. If the answer isn’t in them, no amount of prompt tuning will save you.
◆ Takeaway
Fix retrieval before touching the prompt. Ninety percent of hallucinations start there.