Should we monitor AI system inputs as well as outputs?
Answered
Yes. Monitor outputs and performance for sudden or gradual security-relevant change, and — in line with privacy and data protection — log inputs such as prompts and queries to support audit, investigation and remediation. Secure by Design also requires detect-and-respond capabilities.
From the guidance
Primary (how) Guidelines for secure AI system development
In line with privacy and data protection requirements, you monitor and log inputs to your system (such as inference requests, queries or prompts) to enable compliance obligations, audit, investigation and remediation in the case of compromise or misuse.
Section: Secure operation and maintenance
Read this in Guidelines for secure AI system development (opens in new tab)
Secondary (normative) Secure by Design Principles
Integrate appropriate security logging, monitoring, alerting and response capabilities.
Section: 5. Build in detect and respond security
Related questions
- How do I monitor an AI system once it is live?
- How should I threat-model an AI system? Security and safe use of tools
- What is data poisoning in an AI system? Security and safe use of tools
- Who is accountable if an AI system causes harm or makes a bad decision?
- How should I secure the AI supply chain (models and components)?