← All papers

Ecaveo Working Papers, AI Adoption: The Higher-Order Issues

Approved Once, Changed Daily

from Point-in-Time Ethics to Continuous AI Assurance

Paul Forrest

Fractional Head of AI, Ecaveo

Published
May 2023
Pages
30
Topic
Assurance and audit
Type
Working paper

Abstract

A single approval of an adaptive AI system certifies a system that will soon not exist, and without defined triggers for looking again the approval becomes a record whose main beneficiary is the organisation's own file rather than the people affected. This paper identifies four routes by which an approved system changes, being model updates, prompt and configuration changes, new data sources and new uses, and observes that only the first is routinely recorded, usually by the supplier. It proposes a nine-element continuous assurance model covering a versioned description, a change log, agreed reassessment triggers, performance and bias monitoring, an incident route open to affected people, renewed notice on a change of purpose, a named owner for supplier changes, and retirement criteria agreed before anyone has an interest in keeping the system running. Six triggers and four proportionality tiers keep the burden matched to the effect on people. The structure is borrowed from medical device regulation, where a system is approved together with an envelope of expected change. Four conditions for research ethics panels are added, since a study of a commercial tool faces the same problem.

Read the paper

Download PDF 0.6 MB · 30 pages · no registration

This is the reading edition, complete and unabridged. The designed edition, with the full illustrated layout, is available from the original publication page.

Cite this paper

Forrest, P. (2023). Approved Once, Changed Daily: from Point-in-Time Ethics to Continuous AI Assurance. Ecaveo Working Papers, AI Adoption: The Higher-Order Issues. Ecaveo Services Ltd. https://papers.ecaveo.com/approved-once-changed-daily-continuous-ai-assurance/

Download RIS Download BibTeX RIS opens in Zotero, Mendeley, EndNote and RefWorks

BibTeX, inline
@techreport{forrest2023approvedonce,
  author      = {Forrest, Paul},
  title       = {Approved Once, Changed Daily: from Point-in-Time Ethics to Continuous AI Assurance},
  institution = {Ecaveo Services Ltd},
  series      = {Ecaveo Working Papers, AI Adoption: The Higher-Order Issues},
  year        = {2023},
  month       = {05},
  pages       = {30},
  url         = {https://papers.ecaveo.com/approved-once-changed-daily-continuous-ai-assurance/}
}