> this kind of hallucination is new and unexpected

I don’t think it’s new or unexpected at all. Remember the lawyers who used ChatGPT and it ended up fabricating case law?

ChatGPT is interesting and all but it’s seriously untrustworthy.