Andrej Karpathy on “hallucinations” being a feature, not a bug. I’ve always been surprised how many writers get annoyed when Subtxt sometimes get things “wrong”. That’s creativity!

At the other end of the extreme consider a search engine. It takes the prompt and just returns one of the most similar “training documents” it has in its database, verbatim. You could say that this search engine has a “creativity problem” - it will never respond with something new. An LLM is 100% dreaming and has the hallucination problem. A search engine is 0% dreaming and has the creativity problem.

When it comes to writing stories, I’ll take the hallucination problem over the creativity problem any day of the week.