The Real AI Risk Is Humans Who Stop Checking

Every major safety-critical system civilization has built — aviation, nuclear, financial risk — operates on the same foundational assumption: failure produces a warning . AI systems, as currently deployed, violate that assumption structurally. The risk Spitz identified is not that an AI pursues a misaligned goal with terrifying competence; it is that an AI produces a wrong answer with terrifying plausibility, and the human in the loop has been trained by months of correct outputs to stop verifying.