Focus on simplicity to control

AI, especially machine learning, is not just a mathematical model for working with algorithms. Rather, it should be seen as a global system, where our behaviors are measured and data takes shape, which is processed by algorithmic models for prediction or decision making. These results are then presented in an interface and used interchangeably by humans.

With AI, the rules have changed

However, unlike what used to happen with computer programs, today it is the machine that alone sets the rules that link the model’s input and output data. It is this ability that gives AI all its power and allows us to solve problems that go beyond human formality. However, this is accompanied by a limitation: it is difficult to understand the rules applied by such and such algorithms to achieve such results.

What happens to the model sometimes remains a mystery, even though many explanatory techniques have progressed in recent years. This is the famous “black box effect”. However, when we decide to use AI to make decisions or make predictions that will affect the lives of citizens, it is very important to keep control of the models and guarantee their interpretability and soon to be regulators.

So the interpretability of AI models is becoming a major issue for those who design and use them. It is a question of knowing which factors affect each decision and prediction, in what order (local interpretation), but which rules govern the model in general (global interpretation).

Behind the explanation of AI, there is the necessity of explanation

Access to these processes will allow data scientists and developers to control their models, ensure the relevance of their decisions, and make corrections if necessary.

This clarity will also be useful for beginners and end users. Imagine a machine learning model applied for medical diagnosis. Both the doctor and the patient need to know by what criteria the machine has concluded a specific pathology. But their expectations will not be exactly the same. Physicians will benefit from explanations to have full confidence in the model’s decision. The patient will want to get an explanation of the results. It is clear that the solution depends on the user, the information and the way it is presented must be different. We touch on the concept of AI’s interpretability here, impossible without explanation.

This ability to understand the rules used by algorithmic models also works for the observer. The rules that are slowly being put in place around AI will control the level of criticism of products marketed in European markets tomorrow. To be approved, their rules must be explanatory.

The degree of interpretability that depends on the model

In the case of general models like decision tree or regression we speak of direct explanatory. In their mathematical formulas, the latter provides the importance (weight) of the various factors used by the model to make direct predictions or decisions. In the banking and insurance sectors, for example, the law already needs to provide this direct interpretation, and therefore adopts general models. If Mr. Dupont refuses his loan, the banker must be able to explain the reason to him.

Indirect explanations relate to more complex models such as ensemble models (random forest, gradient boosting, etc.), SVMs (support vector machines) or even neural networks (deep learning), none of which can be directly explained. However, there are open source tools, such as LIME or SHAP, that can now estimate the importance of the various factors that have entered into the model’s rules.

In the case of neural networks used for image or text analysis, new solutions are regularly appearing, thanks to the extensive research around black boxes semi-open solitude or attention maps. We are now able, for example, to identify which areas the model has focused its “attention” on in an image to make its decision and thus, to verify its relevance.

Understand algorithm decisions at all levels

Explanatoryness may be local, that is, an attempt to define for each prediction or decision, the factors were decisive. The so-called global explanatory concern, for its part, rules the model. Both of these approaches make it possible for data scientists to provide information about the robustness of the system and to interpret user results.

To illustrate this dual level, let us take the example of a model for decision making in the case of bank loans. Global exposure will tell us what matters, such as income, gender, financial liability or even the age of the borrower; In other words, it gives us the rules of the model. Local explanations will make it possible for each person to know which specific element led to the decision / prediction. If we take the example of Mr. Dupont, the local explanation will be able to tell us that his loan was rejected because he is 75 years old and already has two credits.

However, from a legal point of view, interpretations provided by direct explanations are considered directly verifiable. Indeed, indirect explanations hold a margin of uncertainty, as it is based on conjecture.

Tendency towards economical, explanatory and controllable AI

Since direct explanations are possible only through the use of simple algorithmic models, the tendency of AI is towards a certain economy. The latter is reinforced by the dynamics of data science which tends to be stronger and more economical from an energy perspective.

Some people will regret that this method has been done for the loss of performance search. However, since there is a real distrust among the people today for the automation of certain processes, for fear of replacing human decision-making with artificial intelligence, it seems fundamental to acquire good interpretations of models to enable confidence.

In France, the level of maturity in this regard is very different. While some companies are still storing, sorting and providing their data, others are already setting up high-performance systems without thinking too much about their ethical standards, which is often considered an undisputed luxury.

An expected regulation that goes in the right direction

If all GAFAM today demands the establishment of ethical partnerships and self-regulation, then a clean and shared structure of the AI ​​ecosystem is needed. Only Europe is able to provide it. The way it has made it possible, with the help of GDPR, to better protect the personal data of citizens, tomorrow, it will contribute to regaining confidence in this ecosystem by controlling the vision of a more humane, interpretable AI. And continuity.

Companies should not fear this impending regulation, but should take it as an opportunity to implement AI solutions, ethical by design. Remember that AI works in concert with people who are an integral part of the system. So instead of starting projects in favor of hyperperformance, there is a real consistency between adopting simple and economical models centered around humane, explanatory and transparent focuses, which are more opaque, and whose control and maintenance are often problematic.

Annabelle Blanchero is a Doctor of Neuroscience, a teacher-researcher at Oxford and New York University. He specializes in the analysis of attentive adaptation processes through machine learning algorithms. Today, Annabelle Blanchero is a senior manager and data scientist at Ecmetrics.

Leave a Comment