2026-04-30 11:04 UTC
@lzg @oscarjiminy The original posts I saw on the incident were loaded with anthropomorphization ("the AI 'went rogue'") which is evocative but absolute garbage framing.
HUMANS designed a system with massive single-points-of-failure, HUMANS ran untested and unreviewed automation which had unintended and catastrophic consequences, HUMANS gave admin access to a third-party automation system without having defined the most fundamental requirements and acceptance criteria for it to determine whether it was fit for purpose, HUMANS approved a change process that had no control points to manage risk and prevent cascading failure.
One of the incredibly corrosive aspects of agentic tools is the absolute disregard for QA, change control, and thoughtful process. It's all YOLO all the time. The departments allowing or mandating the use of these tools exercise even less diligence in making sure their tools are fit for purpose than they normally do which is usually next to nothing. If they were doing their jobs, IT would never deploy Excel to non-finance workers (and even then).
What we're seeing is rank incompetence from the CEO and CTO on down through IT and the software/devops organizations - people making catastrophically bad devisions and shrugging off the consequences with "oh, the _AI_ did it..." No, nobody at the top did their job to ensure the money and effort put in resulted in a robust system and effective change control processes. That wasn't the AI's responsibility, it was that of the company's executives and directors.
Replies (1)
-
@oscarjiminy@aus.social 2026-04-30 11:12
@lzg @arclight Venture capital fuelled voraciously greedy assholes marketed a product unfit for purpose to a consumer base primed their entire lives on the underlying mythology of agentic ai that’ll replace a human workforce