Reliability for unreliable LLMs