Humility in AI protects the quality of Decision systems

In DataRobot, Humble AI is a new feature that protects the quality of your predictions in circumstances where the model is not confident enough. With Humble AI, users make rules for models in deployment for forecasts made in real-time. These rules recognize conditions which indicate that a prediction may not be sure and trigger actions such as defaulting to a 'safe' prediction, overriding outlier values, or not predicting at all.

Even if Humble AI is a new concept, it is clear that it holds tremendous value for decision systems. It is particularly real-time decision systems where each decision counts and is urgent. Before we talk about the cost to business decision-making, let's look at the three most common frameworks around AI and human intelligence working as a single integrated decision system.

Human-in-the-loop

In this process, the individual operating the system considers the recommendation of the AI system but takes the final call. For instance, though a doctor signs off on the definitive diagnosis and prognosis of kidney failure, they leverage an advanced visual artificial intelligence system to score the patient's X-rays.

Human-out-of-the-loop

In this process, the AI system is entirely under control with zero human involvement. For example, an AI is running a real-time bidding (RTB) system creates instantaneous predictions around potential ad buys in milliseconds, without any human involvement.

Human-over-the-loop

An individual, in this process, supervises the system and can intervene whenever the AI system runs into an unexpected scenario and deviates from its performance standard. For instance, a regression model monitors manufacturing processes in real-time. As soon as an outlying prediction is triggered, a human operator is alarmed. And it turns off the system and investigates.

When it's about choosing a level of automation for any given system, the resulting trade-off should be considered. On the edge of the spectrum, you have unbridled industrialisation that approaches maximum efficiency. However, it comes at the cost of losing the guardrails delivered by humans looped into the process. If your appetite is compact or you're working in a highly regulated environment, the human-out-of-the-loop system may cause a serious business risk.

However, in some cases, such as real-time ad bidding, it's impossible to get a human in the loop with decision-making capacity in a fraction of second. This is all accomplished based on a head-spinning amount of anonymized user information and quick behavioural patterns of each user. Such digital advertising systems can multi-task and effectively when a human is not in the loop. How do you mitigate risks when the decision window is unexpectedly small, and the decisions are extraordinarily complicated?

This is when the human-over-the-loop and humility in AI concept come into play. Human-over-the-loop system can balance between automation and human participation by enabling the system to perform in the fully automated mode with human intervention when required.

DataRobot MLOps currently supports a long-term oversight framework by uninterrupted collecting scoring data, predictions, and original outcomes to compare trained and installed models' statistical properties. Wherein it can help detect the critical moment when retraining the model is necessary, humility in AI indicates that human intervention should also be an alternative on the level of an instantaneous individual prediction. However, there are a few primary questions in it.

How to understand when precisely human beings should be involved? It depends on the confidence level of the model's prediction. The confidence level could be determined by monitoring:

Uncertainty in predictions: Predicted values are beyond the range of expected values

Outlying outputs: There are several features in the scoring data dissimilar to what the training model absorbed

Low monitoring regions: A categorical feature value is the one which a user has mentioned unexpected or inappropriate.

With DataRobotMLOps, capabilities such as no operation, overriding prediction, and discarding the prediction are baked into the Humble AI feature.