Finding the identities of missing migrants using artificial intelligence

Every year, thousands of people leave their homes in search of a better life or to flee violence. Many were injured or killed on the way. Many have lost loved ones without realizing they are alive or dead.

According to the International Organization for Migration’s (IOM) Missing Migrants Project, we have lost 45,000 migrants worldwide since 2014, including 24,000 in the Mediterranean.

In 2020, the trans-regional forensic team of the International Committee of the Red Cross (ICRC) was contacted with the INSA (National Institute of Applied Sciences) alliance, with the aim of improving the process of identifying dead migrants in the Euro-Mediterranean region. . Many have drowned here – 16,000 since 2014. To our knowledge, this effort, led by ICRC anthropologist Jose Pablo Barrebar, is the only way to address this problem in the entire region.

INSA teams thus intervened to offer solutions to the ICRC’s essential identification work, which has to deal with a large amount of scattered or substandard information on missing persons.

The partnership took shape after a pilot project led by INSA Lyon, which provided the ICRC with equipment to handle the information on recovered bodies. He joined the INSA Foundation’s Alliance Program.

The program brings together students and faculty-researchers in areas that require scientific and technical expertise from NGOs, such as Handicap International or ICRC. In all, there are 37 students who have created seven projects as part of their course, combining specific methods and tools for engineering schools with knowledge of ICRC’s field.

Artificial intelligence in the service of humanism

Theoretically, the process of identifying drowned people can be easily started by identifying the dead person using their relatives photograph. However, these documents are not always “shown”: either these photos are of poor quality, or the bodies are so damaged and the images are so traumatic that they prevent any official recognition.

This situation led us to explore the idea of ​​comparing photos of the deceased with photographs of people who wanted their relatives using facial recognition technology.

This method was specifically explored in 2020 as part of Jacqueline Halloween’s end-study internship. His project involved the use of facial recognition algorithms and models in the identification of drowned human remains and then evaluated.

In concrete terms, it involves adapting and using models Machine learning, An artificial intelligence strategy that allows a program to learn to recognize similarities and differences between data sets on its own. By dealing with repeated experiences, such as identifying a person’s identity, the program provides training and improves its results. This work has made it possible to verify the interest in this strategy for the recognition of absentees.

To implement this, we compared pictures of living immigrants with dead immigrants in the hope of finding positive matches. For this, we set up a similarity index based on a combined algorithm that makes it possible to obtain a person’s potential identity score in the form of percentages.

Everything is integrated into a web application for people with legal responsibilities for identifying bodies, such as ICRC agents and forensic institutes. This application is under development and the goal of each project is to improve it.

The results obtained are encouraging. Thanks to this software, we’ve been able to create a complete prototype of applied facial recognition for missing immigrants. However, to be able to provide truly reliable indicators of the similarity between the photos of living and dead people, one has to get thousands of photos.

Demonstration video of facial retouching software developed by Insa Network for ICRC (INSA / CICR).

Having set these limits, the tool created today provides ICRC agents with a list of possible matches and offers the possibility to conduct their searches, the search is certainly laborious, but humanely possible.

Continuous software development

At the beginning of this project, in 2020, specifications had to be drawn. INSA students and their teacher Charles Dosal therefore translated into technical language the automatic or non-automatic processing that would take place in these images: remove the face from the decor (a bag, the bottom of a boat, a table, etc.), center the image and align, the wounds Reduce or remove, remove the foam from the face and give the face a glimpse of life.

4 in two studentse Years, Adam Hamidullah and Day Trim Fun, then programmed the algorithms that we identified as the most relevant to solve these various problems. Sometimes parts of healthy skin needed to be “digitally healed” by copying the wound or inserting the eye from another mouth when it was very damaged. The results are encouraging, but we have also been able to measure whether artificial intelligence (AI) can provide more successful answers.

During the summer of 2021, Zoé Philippon and Jeong Hwan Ko saw these awesome images to see more precisely what AI could bring to this mission.

Zoé Philippon’s goal was to examine the limits of facial recognition algorithms based on artificial neural networks when applied to the face image of a dead person, mainly of African descent. These algorithms are just as effective as the images used to calibrate them, with living human faces, mostly white, and a small proportion of male, female or African faces.

So he conducted numerous tests, retraining AI to be more effective in missing persons images. The results indicate that these algorithms will benefit from being more specifically trained in the face of a population that is more representative of the absentee, and that recognition becomes significantly worse when a recognized person dies. Access to a larger amount of data can ensure this very encouraging initial result.

Digital makeup

Jeong Hwan Ko has tried to improve the results of “digital make-up” using artificial neural networks, also pre-trained to fill in the gaps in the image. These methods have proven to be extremely effective in removing injuries, but to repair a face or eye, it was necessary to use other neural networks capable of inserting parts of one image into another.

At the moment the programmer chooses the image to insert, but in the future it will probably be more effective to allow the algorithm to find a large database, eye, mouth or ear in good condition which is similar to most images. The face needs to be identified. There is still work to be done and, here again, extensive access to data will undoubtedly improve the quality of this face reconstruction.

Today, projects continue. We are always looking for data to further train machine learning programs. We are also looking for corporate sponsors willing to share their technology, time and support with us.

Finally, it should be noted that these same applications, designed to respond to the crisis of missing migrants, can also be used in other contexts, such as disasters, conflicts, or in situations where the identification of dead persons may be the cause.


This article is co-authored by Samuel Kenny, Coordinator of the ICRC-INSA Alliance.

Leave a Comment