Should we red-team AI before release?

Answered

Yes for security evaluation. NCSC says release only after appropriate security evaluation such as benchmarking and red teaming, and be clear about known limitations. The ADM framework also recommends red-team testing for algorithmic systems.

From the guidance

Primary (how) NCSC: Secure deployment (AI systems)

You release models, applications or systems only after subjecting them to appropriate and effective security evaluation such as benchmarking and red teaming (as well as other tests that are out of scope for these guidelines, such as safety or fairness), and you are clear to your users about known limitations or potential failure modes.

Section: Release AI responsibly

Read this in NCSC: Secure deployment (AI systems) (opens in new tab)

Related questions

• Should we red-team automated decision-making systems before go-live? Lawful, ethical and responsible use
• What skills and roles do I need on an AI project team? Getting started
• Should we monitor AI system inputs as well as outputs?
• Do I need to tell users when a service uses AI?
• Do staff need AI-specific cyber security training?