One of the near-universal complaints about AI tools is that they can make up information and present it as if it was “true” (aka hallucinate).
Behind the scenes of all of these model, tool, and feature releases, research is being done to address issues like hallucinations. Teams from Anthropic and ETH Zurich have been working towards real-time hallucination detection and their work is promising.
There are a lot of challenges here -
- Sometimes “true” depends on context that can change
- What is “true” for one person (or in one situation) may not be “true” for another
- Important context is often left out of input prompts
- AI tools use internet sites to validate information - and we know that everything on the internet is true
- Etc. etc.
Despite these and other challenges, the research has shown impressive results with real-time detection of information that the models are uncertain about. This can help people save a lot of time when fact-checking important output. Models were able to identify hallucinated information with up to a 90% accuracy in testing.
Now it’s still early and there is still a lot of testing to be done, but these results are promising.
This is a shortened version of a fairly dense report. If you want all of the details, here is the full report - Real-Time Detection of Hallucinated Entities in Long-Form Generation
Comments