Untracked prompt changes cause hallucination rate surge from 2% to 15%
criticalconfigurationUpdated Mar 9, 2026(via Exa)
Technologies:
How to detect:
Without prompt version management, a prompt change caused hallucination rate to surge from 2% to 15% for inputs in a specific language (Japanese). With no prompt change history, rollback took 3 hours, during which hundreds of incorrect responses were served.
Recommended action:
Implement prompt version management using LangFuse's Prompt Management feature with versions and labels (production, staging). Reference prompts by name and version using langfuse_prompt_name parameter. Each trace automatically records which prompt version was used, enabling rapid rollback and A/B testing.