How do I measure the impact of an AI solution?

Answered

Plan evaluation early with a baseline and Theory of Change. Measure benefit/ROI, quality, efficiency, satisfaction, usage and cost during development and after release. After release you must evaluate regularly to confirm the solution still meets user needs.

From the guidance

Primary (how) AI Knowledge Hub: Measuring the impact of AI solutions

Make sure impact evaluation is included in the design of your product throughout its life cycle, for example: plan your evaluation strategy early so you can align delivery with continuous feedback; include evaluation metrics in objectives and key results; develop a Theory of Change; establish a baseline so you can compare information from before the solution was implemented.

Section: How to approach impact evaluation

Read this in AI Knowledge Hub: Measuring the impact of AI solutions (opens in new tab)

Secondary (normative) AI Knowledge Hub: Measuring the impact of AI solutions

After release you must do regular evaluations to understand overall impact make sure the model is continuing to meet user needs

Section: Measure impact after release

Read this in AI Knowledge Hub: Measuring the impact of AI solutions (opens in new tab)

Related questions