Indeed, LLM's seem to be much worse at introspection than humans. I wonder what would happen if one used reinforcement learning to train into it the ability to correctly predict and reason about it's capabilities and behavior.
(Uses recurrent langchain loops for introspection and learning about itself and its capabilities as they grow + vector databases like Pinecone for long term memory)
