Out-Of-Distribution Detection Is Not All You Need
Joris Guérin, Kévin Delmas, Raul Ferreira, Jérémie Guiochet
AAAI Conference on Artificial Intelligence (AAAI 2023)
[PDF] [arXiv] [bibtex]

The usage of deep neural networks in safety-critical systems is limited by our ability to guarantee their correct behavior. Runtime monitors are components aiming to identify unsafe predictions and discard them before they can lead to catastrophic consequences. Several recent works on runtime monitoring have focused on out-of-distribution (OOD) detection, i.e., identifying inputs that are different from the training data. In this work, we argue that OOD detection is not a well-suited framework to design efficient runtime monitors and that it is more relevant to evaluate monitors based on their ability to discard incorrect predictions. We call this setting out-of-model-scope detection and discuss the conceptual differences with OOD. We also conduct extensive experiments on popular datasets from the literature to show that studying monitors in the OOD setting can be misleading: 1. very good OOD results can give a false impression of safety, 2. comparison under the OOD setting does not allow identifying the best monitor to detect errors. Finally, we also show that removing erroneous training data samples helps to train better monitors.

Unifying Evaluation of Machine Learning Safety Monitors
Joris Guérin, Raul Ferreira, Kévin Delmas, Jérémie Guiochet
33rd IEEE International Symposium on Software Reliability Engineering (ISSRE 2022)
[PDF] [arXiv] [bibtex]

With the increasing use of Machine Learning (ML) in critical autonomous systems, runtime monitors have been developed to detect prediction errors and keep the system in a safe state during operations. Monitors have been proposed for different applications involving diverse perception tasks and ML models, and specific evaluation procedures and metrics are used for different contexts. This paper introduces three unified safety-oriented metrics, representing the safety benefits of the monitor Safety Gain, the remaining safety gaps after using it Residual Hazard, and its negative impact on the system's performance Availability Cost. To compute these metrics, one requires to define two return functions, representing how a given ML prediction will impact expected future rewards and hazards. Three use-cases (classification, drone landing, and autonomous driving) are used to demonstrate how metrics from the literature can be expressed in terms of the proposed metrics. Experimental results on these examples show how different evaluation choices impact the perceived performance of a monitor. As our formalism requires us to formulate explicit safety assumptions, it allows us to ensure that the evaluation conducted matches the high-level system requirements.

Evaluation of Runtime Monitoring for UAV Emergency Landing
Joris Guérin, Kévin Delmas, Jérémie Guiochet
International Conference on Robotics and Automation (ICRA 2022)
[PDF] [arXiv] [bibtex]

To certify UAV operations in populated areas, risk mitigation strategies -- such as Emergency Landing (EL) -- must be in place to account for potential failures. EL aims at reducing ground risk by finding safe landing areas using on-board sensors. The first contribution of this paper is to present a new EL approach, in line with safety requirements introduced in recent research. In particular, the proposed EL pipeline includes mechanisms to monitor learning based components during execution. This way, another contribution is to study the behavior of Machine Learning Runtime Monitoring (MLRM) approaches within the context of a real-world critical system. A new evaluation methodology is introduced, and applied to assess the practical safety benefits of three MLRM mechanisms. The proposed approach is compared to a default mitigation strategy (open a parachute when a failure is detected), and appears to be much safer.

Combining pretrained CNN feature extractors to enhance clustering of complex natural images
Joris Guérin, Stéphane Thiery, Éric Nyiri, Olivier Gibaru, Byron Boots
Neurocomputing, 2021
[PDF] [arXiv] [bibtex]

Recently, a common starting point for solving complex unsupervised image classification tasks is to use generic features, extracted with deep Convolutional Neural Networks (CNN) pretrained on a large and versatile dataset (ImageNet). However, in most research, the CNN architecture for feature extraction is chosen arbitrarily, without justification. This paper aims at providing insight on the use of pretrained CNN features for image clustering (IC). First, extensive experiments are conducted and show that, for a given dataset, the choice of the CNN architecture for feature extraction has a huge impact on the final clustering. These experiments also demonstrate that proper extractor selection for a given IC task is difficult. To solve this issue, we propose to rephrase the IC problem as a multi-view clustering (MVC) problem that considers features extracted from different architectures as different “views” of the same data. This approach is based on the assumption that information contained in the different CNN may be complementary, even when pretrained on the same data. We then propose a multi-input neural network architecture that is trained end-to-end to solve the MVC problem effectively. This approach is tested on nine natural image datasets, and produces state-of-the-art results for IC.

Robust Detection of Objects under Periodic Motion with Gaussian Process Filtering
Joris Guérin, Anne Magaly de Paula Canuto, Luiz Marcos Garcia Gonçalves
International Conference on Machine Learning and Applications (ICMLA), 2020
[PDF] [arXiv] [bibtex] [video presentation]

Object Detection (OD) is an important task in Computer Vision with many practical applications. For some use cases, OD must be done on videos, where the object of interest has a periodic motion. In this paper, we formalize the problem of periodic OD, which consists in improving the performance of an OD model in the specific case where the object of interest is repeating similar spatio-temporal trajectories with respect to the video frames. The proposed approach is based on training a Gaussian Process to model the periodic motion, and use it to filter out the erroneous predictions of the OD model. By simulating various OD models and periodic trajectories, we demonstrate that this filtering approach, which is entirely data-driven, improves the detection performance by a large margin.

Semantically Meaningful View Selection
Joris Guérin, Olivier Gibaru, Éric Nyiri, Stéphane Thiery, Byron Boots
International Conference on Intelligent Robots and Systems (IROS), 2018
[PDF] [arXiv] [bibtex] [presentation] [data]

An understanding of the nature of objects could help robots to solve both high-level abstract tasks and improve performance at lower-level concrete tasks. Although deep learning has facilitated progress in image understanding, a robot's performance in problems like object recognition often depends on the angle from which the object is observed. Traditionally, robot sorting tasks rely on fixed top-down views of the objects. By changing its viewing angle, a robot can select a more semantically informative view leading to better performance for object recognition. In this paper, we introduce the problem of semantic view selection, which consists in finding good camera poses to gain semantic knowledge about observed objects. We propose a conceptual generic formulation of the problem, together with a relaxation based on clustering, to make it solvable. We then present a new image dataset consisting of around 10k images representing various views of 144 objects under different poses. Finally we use this dataset to propose a first solution to the problem by training a neural network to predict a "semantic score" from a top view image and camera pose. The views predicted to have higher scores are then showed to provide better clustering results than fixed top-down views.

Machine learning improvements for robotic applications in industrial context: Case study of autonomous sorting
Joris Guérin
Ph.D. dissertation (2018)
[PDF] [bibtex] [presentation]

Thanks to their flexible mechanical design, modern industrial robots can be programmed for different tasks without physical modification. In addition, they are highly instrumented and should be able to be responsive to their environment. However, the use of robots in industry is still restricted to repeatable tasks with low level of adaptability. In an industrial context, it is essential to program robots that can autonomously adapt to different applications and are robust to changes in working conditions. The machine learning framework for robot programming is well suited to design such kinds of adaptive and robust applications. Hence, in this thesis, several machine learning contributions are presented, aiming at designing smarter robotic applications, with a broader operational range. The methods developed are centered on autonomous sorting, but may be useful to address problems in many other subfields of robotics. Throughout this thesis, we propose new approaches to image clustering, optimal view selection, trajectory learning and stereo localization, with the objective of designing more universal robotic sorting applications.