In recent years, the emergence of powerful computational resources and the proliferation of extensive annotated datasets
have catalyzed significant breakthroughs in Machine Learning.
This has empowered us to solve previously intractable tasks involving complex data types like images and text.
However, despite their great merits, these methods can fail when operating outside their ideal environments,
especially in scenarios with scarce, low-quality, or partially annotated data, or when the operational context diverges from the training context.
My research aims to develop innovative learning approaches, crafted for more restrictive application contexts.
The primary objective is to optimize available resources, such as time, expertise, and funding, while
ensuring sufficient performance to meet the goals of the target application.
A unique aspect of my work lies in the exploration of the intricate relationship between a machine learning model
and the overarching system within which it operates.
Through applications in diverse domains, my investigations have emphasized the criticality of understanding this holistic integration,
throughout all the development phases, from initial data acquisition to final deployment.
Main Contributions by Research Areas
Neural Network Runtime Monitoring
New dependability challenges arise when Machine Learning models are used in safety-critical applications
like autonomous cars and surgical robots. Runtime Monitors are fault tolerance mechanisms aiming to ensure
the safe behavior of the system despite the occurrence of prediction errors, notably through better estimation of
prediction uncertainty. Our recent contributions have highlighted several shortcomings about how the performance
of current methods is assessed, and proposed new generic evaluation frameworks, better suited to the diverse set of applications
where runtime monitoring is used. Our works were presented at top conferences in robotics, safety and artificial intelligence,
which underlines the cross-domain relevance of our research in this area.
Out-Of-Distribution Detection Is Not All You Need
Joris Guérin, Kévin Delmas, Raul Ferreira, Jérémie Guiochet
AAAI Conference on Artificial Intelligence (AAAI 2023)
The usage of deep neural networks in safety-critical systems is limited by our ability to guarantee their
correct behavior. Runtime monitors are components aiming to identify unsafe predictions and discard them
before they can lead to catastrophic consequences. Several recent works on runtime monitoring have focused
on out-of-distribution (OOD) detection, i.e., identifying inputs that are different from the training data.
In this work, we argue that OOD detection is not a well-suited framework to design efficient runtime monitors
and that it is more relevant to evaluate monitors based on their ability to discard incorrect predictions.
We call this setting out-of-model-scope detection and discuss the conceptual differences with OOD.
We also conduct extensive experiments on popular datasets from the literature to show that studying
monitors in the OOD setting can be misleading: 1. very good OOD results can give a false impression of safety,
2. comparison under the OOD setting does not allow identifying the best monitor to detect errors.
Finally, we also show that removing erroneous training data samples helps to train better monitors.
Unifying Evaluation of Machine Learning Safety Monitors
Joris Guérin, Raul Ferreira, Kévin Delmas, Jérémie Guiochet
33rd IEEE International Symposium on Software Reliability Engineering (ISSRE 2022)
With the increasing use of Machine Learning (ML) in critical autonomous systems, runtime monitors
have been developed to detect prediction errors and keep the system in a safe state during operations.
Monitors have been proposed for different applications involving diverse perception tasks and ML models,
and specific evaluation procedures and metrics are used for different contexts. This paper introduces three
unified safety-oriented metrics, representing the safety benefits of the monitor Safety Gain,
the remaining safety gaps after using it Residual Hazard, and its negative impact on the system's
performance Availability Cost. To compute these metrics, one requires to define two return functions,
representing how a given ML prediction will impact expected future rewards and hazards.
Three use-cases (classification, drone landing, and autonomous driving) are used to demonstrate
how metrics from the literature can be expressed in terms of the proposed metrics. Experimental results
on these examples show how different evaluation choices impact the perceived performance of a monitor.
As our formalism requires us to formulate explicit safety assumptions, it allows us to ensure that the
evaluation conducted matches the high-level system requirements.
Evaluation of Runtime Monitoring for UAV Emergency Landing
Joris Guérin, Kévin Delmas, Jérémie Guiochet
International Conference on Robotics and Automation (ICRA 2022)
To certify UAV operations in populated areas, risk mitigation strategies -- such as Emergency Landing (EL) --
must be in place to account for potential failures. EL aims at reducing ground risk by finding safe
landing areas using on-board sensors. The first contribution of this paper is to present a new EL approach,
in line with safety requirements introduced in recent research. In particular, the proposed
EL pipeline includes mechanisms to monitor learning based components during execution. This way, another
contribution is to study the behavior of Machine Learning Runtime Monitoring (MLRM)
approaches within the context of a real-world critical system. A new evaluation methodology is
introduced, and applied to assess the practical safety benefits of three MLRM mechanisms.
The proposed approach is compared to a default mitigation strategy
(open a parachute when a failure is detected), and appears to be much safer.
Transfer Learning - Optimizing the use of publicly available data and models
Combining pretrained CNN feature extractors to enhance clustering of complex natural images
Joris Guérin, Stéphane Thiery,
Recently, a common starting point for solving complex unsupervised image classification tasks is to use generic features,
extracted with deep Convolutional Neural Networks (CNN) pretrained on a large and versatile dataset (ImageNet). However,
in most research, the CNN architecture for feature extraction is chosen arbitrarily, without justification. This paper aims
at providing insight on the use of pretrained CNN features for image clustering (IC). First, extensive experiments are conducted
and show that, for a given dataset, the choice of the CNN architecture for feature extraction has a huge impact on the final clustering.
These experiments also demonstrate that proper extractor selection for a given IC task is difficult. To solve this issue,
we propose to rephrase the IC problem as a multi-view clustering (MVC) problem that considers features extracted from different
architectures as different “views” of the same data. This approach is based on the assumption that information contained in the
different CNN may be complementary, even when pretrained on the same data. We then propose a multi-input neural network architecture
that is trained end-to-end to solve the MVC problem effectively. This approach is tested on nine natural image datasets,
and produces state-of-the-art results for IC.
Data acquisition and predictions complementarity
Robust Detection of Objects under Periodic Motion with Gaussian Process Filtering
Anne Magaly de Paula Canuto,
Luiz Marcos Garcia Gonçalves
International Conference on Machine Learning and Applications (ICMLA), 2020
Object Detection (OD) is an important task in Computer Vision with many practical applications.
For some use cases, OD must be done on videos, where the object of interest has a periodic motion.
In this paper, we formalize the problem of periodic OD, which consists in improving the performance
of an OD model in the specific case where the object of interest is repeating similar spatio-temporal
trajectories with respect to the video frames. The proposed approach is based on training a
Gaussian Process to model the periodic motion, and use it to filter out the erroneous predictions
of the OD model. By simulating various OD models and periodic trajectories, we demonstrate that
this filtering approach, which is entirely data-driven, improves the detection performance by a large margin.
Semantically Meaningful View Selection
International Conference on Intelligent Robots and Systems (IROS), 2018
An understanding of the nature of objects could help robots to solve both
high-level abstract tasks and improve performance at lower-level concrete tasks.
Although deep learning has facilitated progress in image understanding, a robot's
performance in problems like object recognition often depends on the angle
from which the object is observed. Traditionally, robot sorting tasks rely on
fixed top-down views of the objects. By changing its viewing angle, a robot
can select a more semantically informative view leading to better performance
for object recognition. In this paper, we introduce the problem of semantic
view selection, which consists in finding good camera poses to gain semantic
knowledge about observed objects. We propose a conceptual generic formulation
of the problem, together with a relaxation based on clustering, to make it
solvable. We then present a new image dataset consisting of around 10k images
representing various views of 144 objects under different poses. Finally we use
this dataset to propose a first solution to the problem by training a neural
network to predict a "semantic score" from a top view image and camera pose.
The views predicted to have higher scores are then showed to provide better
clustering results than fixed top-down views.
Machine learning improvements for robotic applications in industrial context:
Case study of autonomous sorting
Ph.D. dissertation (2018)
Thanks to their flexible mechanical design, modern industrial robots can be programmed
for different tasks without physical modification. In addition, they are highly
instrumented and should be able to be responsive to their environment. However,
the use of robots in industry is still restricted to repeatable tasks with low level
of adaptability. In an industrial context, it is essential to program robots that can
autonomously adapt to different applications and are robust to changes in working
conditions. The machine learning framework for robot programming is well suited to
design such kinds of adaptive and robust applications. Hence, in this thesis, several
machine learning contributions are presented, aiming at designing smarter robotic
applications, with a broader operational range. The methods developed are centered on
autonomous sorting, but may be useful to address problems in many other subfields
of robotics. Throughout this thesis, we propose new approaches to image clustering,
optimal view selection, trajectory learning and stereo localization, with the objective
of designing more universal robotic sorting applications.