Singapore’s AI Governance Framework

(link)

This is currently the main text on AI ethics and governance published by a Singaporean entity. I anticipate other reports to follow from the recently established SMU Centre for AI and Data Governance. It is interesting to observe the very pragmatic language used in this framework, even when compared to other guidelines. This probably stems primarily from its objective as a “ready-to-use tool”. This is similar to the assessment section in Part 3 of the European Commission’s Guidelines. Another point to note is that the report appears to be primarily targeted at companies, notifying them about the regulations to follow in the use of AI.

The Model AI Governance Framework (“Model Framework”) focuses primarily on four broad areas: internal governance, decision-making models, operations management and customer relationship management.

It is important to note that the Framework does not seek to “[articulate] a set of ethical principles for AI”. Instead, this Framework is

a general, ready-to-use tool to enable organisations that are deploying AI solutions at scale to do so in a responsible manner

As such, the Framework focuses more on pragmatic measures and less on ethical discussions.

Guiding Principles

The Framework focuses on two main “high-level guiding principles”:

Organisations using AI in decision-making should ensure that the decision-making process is explainable, transparent and fair.

AI solutions should be human-centric.

Key Areas

The Framework highlights the following four key areas:

  • Internal Governance Structures and Measures
  • Determining AI Decision-Making Model
  • Operations Management
  • Customer Relationship Management

Internal Governance Structure and Measures

  • Clear roles and responsibilities for the ethical deployment of AI

This includes a range of roles throughout the AI pipeline, including: management of risk, choosing of algorithms, data selection, training of models, maintenance of AI systems, monitoring feedback, training of staff etc.

  • Risk management and internal controls

This measure considers the issue of biased data, explainability and transparency (including the reporting of confidence levels rather than just predictions), as well as possible issues due to other factors such as manpower turnover.

Determining AI Decision-Making Model

This section is devoted to the general design of the AI system, and begins by proposing three main models (rearranged here in order of descending human control):

  • Human-in-the-loop This refers to a human retaining full control, while the AI system might offer suggestions
  • Human-over-the-loop This refers to humans having the option to adjust parameters and maintaining partial control over the system
  • Human-out-of-the-loop This refers to the AI system having full control, without human override

The section also proposes a simple matrix that classifies tradeoffs between severity of harm and probability of harm. There is no explicit mapping between the matrix and the three models, although it is implied that human control should correlate with the severity and probability of harm.

Operations Management

This section considers the details in an AI pipeline, specifically the data preparation and the choice of algorithms and models.

On datasets

The Framework considers several aspects:

  • Understanding the lineage of data (provenance)
  • Ensuring data quality (including accuracy, completeness, credibility, recency, integrity, relevance, usability and human-intervention)
  • Minimising inherent bias (explicitly considering both selection bias and measurement bias and suggesting diverse sources as a way of mitigating bias)
  • Different datasets for training, testing, and validation
  • Periodic reviewing and updating of datasets

On algorithms and models

This section focuses on practical considerations in the concepts of explainability, robustness, reproducibility, safeguards (described as exception handling), traceability and auditability. Other important concepts include graceful degradation, the use of counterfactual explanations, the distinction between technical transparency and intuitive explainability, and the importance of similar training and testing / deployment environments.

The proposed measures, while more specific and pragmatic compared to other guidelines, fall short of enumerating possible algorithms, tools and resources for AI practitioners. This might be attributed to the neutral stance of the report, with regards to particular commercial entities, as well as the hopefully evergreen nature of the report versus the constantly changing landscape of resources. There is perhaps room for an open-source guide that details these resources and one that can be updated by the public.

Customer Relationship Management

This section details the customer side of the AI pipeline and focuses on concepts of explainability, appropriate disclosure, usability. It also touches on the notion of opting out, feedback channels and decision review channels.

Of particular interest is the list of readability metrics - “the Fry readability graph, the Gunning Fog Index, the Flesh-Kincaid readability tests”, which should also prove useful in my research project.