The end of hallucination (for those who can afford it)? [R]

DeepMind just published a paper about fact-checking text:

https://preview.redd.it/zsmv0a0293rc1.png?width=1028&format=png&auto=webp&s=789c1c2f9b31aa734a7ebcf459df3ad06bd74285

The approach costs $0.19 per model response, using GPT-3.5-Turbo, which is cheaper than human annotators, while being more accurate than them:

https://preview.redd.it/ob7bb3iv73rc1.png?width=1014&format=png&auto=webp&s=e79bbcaa578b29772cb3b43ead508daff7288091

They use this approach to create a factuality benchmark and compare some popular LLMs.

Paper and code: https://arxiv.org/abs/2403.18802

EDIT: Regarding the title of the post: Hallucination is defined (in Wikipedia) as "a response generated by AI which contains false or misleading information presented as fact.": Your code that does not compile is not, by itself, a hallucination. When you claim that the code is perfect, that's a hallucination.