Tech Explained: Here’s a simplified explanation of the latest technology update around Tech Explained: What the UK’s AI Security Institute means for buyers, not builders in Simple Termsand what it means for users..
The establishment by the UK government of the AI Security Institute (AISI) may initially have been seen by buyers and procurement teams as a form of reassurance, giving them easy indicators to know which AI frontier models have undergone government-led safety evaluation. But how an AI service behaves still sits firmly with the deployer. This means organisations must continue to conduct their own due diligence to ensure that models and associated services operate as intended.
The AISI uses advanced safety tools to test for ways to bypass model guardrails, including pre-deployment testing, red-teaming and evaluations for edge cases not found in regular testing. Its remit, however, is narrow, and has shrunk further since last year with the change in name from AI Safety Institute to AI Security Institute, indicating a focus on security-related risks rather than algorithmic bias or freedom of speech.
What do AI buyers need to do?
For buyers, the AISI does not assess the affects of a frontier AI model to specific organisations, deployments and industries. It also does not provide guidance on how an organisation should deploy an AI safely, or mitigate any of the risks brought up in its assessment of a frontier AI model. This means deployers of AI systems are liable for anything that goes wrong during deployment.
While the AISI can provide a “trusted vendor” stamp for these models, it functions more like a safety standards issuer for power tools. If the buyer then goes and drills into a live cable or fails to wear protective equipment, they are liable for the consequences of those actions.
Unlike a power tool, some of the risks of AI deployment are not as obvious. Buyers need to be aware of regulatory compliance, with adherence to regional and national regulations and protections, alongside data privacy and the correct processing of data. Understanding how an AI will interact during an outage or incorrect input can avoid costly reputational damage. Organisations must also keep track of model drift to ensure performance stability.
AI models require constant monitoring
Some of this observation occurs after deployment, with regulators changing the rules around AI frequently. Monitoring for outages and model drift is a continuous process. Buyers need to be keenly aware of the life-cycle of each model, and ensure that they have an active AI model provider that is pushing out regular updates to improve model safety and security.
For buyers, the practical response is not to defer to AISI, but to evolve their own procurement and contracting standards. AISI and other AI security institutes around the world have labelled AI as high-risk software, and so procurement of this software should resemble other security-critical software purchases rather than standard SaaS purchasing processes.
As several legal experts have noted, accountability cannot be outsourced through procurement. Discussing the use of AI in a courtroom, professor Joanna Bryson of the Hertie School, said: “you need to ensure you procure the right kind of AI for a task, and the right kind is determined in part by the essentialness of human responsibility.”
Where AISI can be helpful is getting these AI vendors to provide more feedback, accountability and transparency on model defects and updates. If there are more eyes on the internal workings of a model and more testing performed by neutral actors, then buyers have a far greater quality of information when purchasing.
What this does not eliminate is risk altogether, and the key point is that the AISI has not reduced responsibility, but increased the expectations from all players involved. AISI may filter out the truly bad AI models from procurement cycles, but it also puts the onus even more on buyers to properly test AI models and ensure they are a perfect fit for their own deployment.
The establishment by the UK government of the AI Security Institute (AISI) may initially have been seen by buyers and procurement teams as a form of reassurance, giving them easy indicators to know which AI frontier models have undergone government-led safety evaluation. But how an AI service behaves still sits firmly with the deployer. This means organisations must continue to conduct their own due diligence to ensure that models and associated services operate as intended.
The AISI uses advanced safety tools to test for ways to bypass model guardrails, including pre-deployment testing, red-teaming and evaluations for edge cases not found in regular testing. Its remit, however, is narrow, and has shrunk further since last year with the change in name from AI Safety Institute to AI Security Institute, indicating a focus on security-related risks rather than algorithmic bias or freedom of speech.