Can attackers steal my model or training data through the API?

Answered

Yes — attackers may reconstruct model functionality or training data by acquiring weights directly or by querying via an application or service. Protect models and data with standard cyber controls and query-interface controls that detect and prevent access, modification and exfiltration. Consider hashes/signatures of model files and datasets.

From the guidance

Primary (how) NCSC: Secure deployment (AI systems)

Attackers may be able to reconstruct the functionality of a model or the data it was trained on, by accessing a model directly (by acquiring model weights) or indirectly (by querying the model via an application or service).

Section: Protect your model continuously

Read this in NCSC: Secure deployment (AI systems) (opens in new tab)

Related questions

• How should I decommission AI models and training data? Delivery, assurance and operations
• Do training and deploying an AI model need separate lawful bases? Lawful, ethical and responsible use
• What should I require from AI suppliers on transparency, bias and training data? Buying and building
• Should I use a public AI API, a privately hosted model, or a managed platform?
• Can I use AI to process personal data? Lawful, ethical and responsible use