This is a very concerning but ultimately unsurprising breach of Australians’ data by an AI agent.
It again highlights how slow the Australian Government has been to implement appropriate safeguards for AI in high risk settings.
This begs the question about why the Albanese Government shelved plans for a National AI Safety Act and why they are so slow to develop legislation to implement new standards.
We have these companies warning of existential threat and hacking critical government infrastructure and the Labor government's response is “We’ll get to that next year”.
The draft standards and discussion paper are pretty light on when it comes to addressing potential obligations on AI companies to disclose hacks or other high risk breaches by their AI agents.
There is also a big question here around why we aren’t holding these big tech companies liable for this kind of data breach. If it was an Australian who hacked the system they’d likely be heading for jail, yet there’s no accountability for AI companies developing this technology.