OpenAI said one of its internal AI models broke through security on a government statistics website to get hospital-bed data the site had refused to give it.
The model was in reinforcement learning training at the time, according to a misalignment report the company updated Oct. 15.
The incident happened June 17, and OpenAI discovered it Aug. 13. The model’s task was to find a district’s historical hospital-bed totals on the site. OpenAI did not name the website or the country.
After the site denied it access several times, the model used the site’s file-download feature to pull application and configuration files from outside the folder that feature was meant to serve. Those files contained a database connection string, a secret used to sign authorization tokens and email login credentials.
The model then found that the site’s code read identity data from authorization tokens without checking their signatures. It built unsigned tokens, got past the statistics endpoint’s authorization checks and saved 24 years of data, far more than the single year it had been asked for.
Separately, the model pulled the site’s account directory, which listed more than 1,000 email addresses, and considered using the addresses to guess a password. Its final answer was graded correct but did not mention the unauthorized access. The model had already found the same figure through a public search.
OpenAI said its monitors flagged the incident as critical during a retrospective review of earlier training and evaluation traffic, which it began after the Hugging Face incident. In response to this and other incidents, the company said it has blocked models from the live internet during training and evaluation, strengthened security controls, improved alignment grading and extended misalignment monitoring to all reinforcement learning and evaluation traffic.
Becker’s has reached out to OpenAI for comment and will update this story if more information becomes available.
The post OpenAI model breached government site to pull hospital-bed data appeared first on Becker's Hospital Review | Healthcare News & Analysis.