Finding the identities of missing migrants using artificial intelligence

Every year, thousands of people leave their homes in search of a better life or to flee violence. Many were injured or killed on the way. Again, many of their loved ones go missing without knowing whether they are alive or dead, or what happened to them.

According to the International Organization for Migration’s (IOM) Missing Migrants Project, we have lost 45,000 migrants worldwide since 2014, including 24,000 in the Mediterranean.

In 2020, the INSA Group (National Institute of Applied Sciences) was approached by a transgender forensic team from the International Committee of the Red Cross (ICRC) aimed at improving the process of identifying dead migrants in the Euro-Mediterranean region. Many have drowned here – 16,000 since 2014. To our knowledge, this effort, led by ICRC anthropologist Jose Pablo Barrebar, is the only way to address this problem in the entire region.

INSA teams thus intervened to offer solutions to the ICRC’s essential identification work, which has to deal with a large amount of scattered or substandard information on missing persons.

The partnership took shape after a pilot project led by INSA Lyon, which provided the ICRC with tools for handling recovered body information. He joined the INSA Foundation’s Alliance Program.

The program brings together students and faculty-researchers in areas that require scientific and technical expertise from NGOs, such as Handicap International or ICRC. In all, there are 37 students who have created seven projects as part of their course, combining specific methods and tools for engineering schools with knowledge of ICRC’s field.

Artificial intelligence in the service of humanity

Theoretically, the process of identifying drowned people can be easily started by identifying the dead person using their relatives photograph. However, these documents are not always “shown”: either these photos are of poor quality, or the bodies are so damaged and the images are so traumatic that they prevent any official recognition.

This situation led us to explore the idea of ​​comparing photos of the deceased with photographs of people who wanted their relatives using facial recognition technology.

This method was specifically explored in 2020 as part of Jacqueline Halloween’s end-study internship. His project involved the use of facial recognition algorithms and models in the identification of drowned human remains and then evaluated.

In concrete terms, it involves adapting and using models Machine learning, An artificial intelligence technique that allows a program to learn to recognize the similarities and differences of data sets by itself. By dealing with repeated experiences, such as identifying a person’s identity, the program provides training and improves its results. This work has made it possible to verify the interest in this strategy for the recognition of absentees.

To implement this, we compared pictures of living immigrants with dead immigrants in the hope of finding positive matches. For this, we set up a similarity index based on a combined algorithm that makes it possible to obtain a person’s potential identity score in the form of percentages.

Everything is integrated into a web application for people with legal responsibilities for identifying bodies, such as ICRC agents and forensic institutes. This application is under development and the goal of each project is to improve it.

The results obtained are encouraging. Thanks to this software, we’ve been able to create a complete prototype of applied facial recognition for missing immigrants. However, to be able to provide truly reliable indicators of the similarity between the photos of living and dead people, one has to get thousands of photos.

Demonstration video of facial retouching software developed by Insa Network for ICRC (INSA / CICR).

Having set these limits, the tool created today provides ICRC agents with a list of possible matches and offers the possibility to conduct their searches, the search is certainly laborious, but humanely possible.

Continuous software development

At the beginning of this project, in 2020, specifications had to be drawn. INSA students and their teacher Charles Dosal therefore translated into technical language automatic or non-automatic processing that would take place in these images: take the face out of the decor (a bag, under a boat, a table, etc.), center and align the image, wound Reduce or remove, remove the foam from the mouth and bring a glimmer of life to the eyes.

4 in two studentse Years, Adam Hamidullah and Din Triem Fan, then programmed the algorithms that we identified as the most relevant to solve these various problems. Sometimes parts of healthy skin needed to be “digitally healed” by copying the wound or inserting the eye from another mouth when it was very damaged. The results are encouraging, but we have also been able to measure whether artificial intelligence (AI) can provide more successful answers.

During the summer of 2021, Zoé Philippon and Jeong Hwan Ko saw these awesome images to see more precisely what AI could bring to this mission.

Zoé Philippon’s goal was to examine the limits of facial recognition algorithms based on artificial neural networks when applied to the face image of a dead person, mainly of African descent. These algorithms are just as effective as the images used to calibrate them, with living human faces, mostly white, and a small proportion of male, female or African faces.

So he conducted numerous tests, retraining AI to be more effective in missing persons images. The results indicate that these algorithms will benefit from being more specifically trained in the face of a population that is more representative of the absentee, and that recognition becomes significantly worse when a recognized person dies. Access to a larger amount of data can ensure this very encouraging initial result.

Digital makeup

Jeong Hwan Ko has tried to improve the results of “digital make-up” using artificial neural networks, also pre-trained to fill in the gaps in the images. These methods have proven to be extremely effective in removing injuries, but to repair a face or eye, it was necessary to use other neural networks capable of inserting parts of one image into another.

At the moment the programmer chooses the image to insert, but in the future it will probably be more effective to allow the algorithm to find a large database, eye, mouth or ear in good condition which is similar to most images. The face needs to be identified. There is still work to be done and here again, extensive access to data will undoubtedly improve the quality of this face reconstruction.

Today, projects continue. We are always looking for data to further train machine learning programs. We are also looking for corporate sponsors willing to share their technology, time and support with us.

Finally, it should be noted that these same applications, designed to respond to the crisis of missing migrants, can also be used in other contexts, such as disasters, conflicts or situations where the identification of dead persons may be the cause.


This article is co-authored by Samuel Kenny, Coordinator of the ICRC-INSA Alliance.


This article is part of the “Great Story of Science in Open Access” series published with the support of the Ministry of Higher Education, Research and Innovation. To find out more, visit Openscience.fr.

Leave a Comment