(M/F) : Thesis on the subject of "Multimodal image fusion"

Job Description

Thesis Subject

The AI-HERBAGE project, part of the PEPR AGROÉCOLOGIE & NUMÉRIQUE
(https://www.pepr-agroeconum.fr/les-projets-finances/traitement-de-donnees-et-modelisation/laureats-aap/ai-herbage) initiative, aims to develop algorithms to improve pasture management using data from various sensors. The primary data sources for vegetation monitoring will be aerial and satellite imagery. This data can be highly heterogeneous in terms of spectral, temporal, and spatial resolution, as well as acquisition geometry, noise, and radiometric response.
One of the project's main tasks is to merge data from different sources to obtain the most comprehensive information possible on vegetation status, in terms of spectral and spatial resolution, while also capturing temporal variations.
Several challenges arise in merging images from heterogeneous sensors:
• Aerial and satellite images are captured from different viewpoints, requiring alignment between images and, when accounting for terrain, pixel-level alignment. Due to the heterogeneity of the sources, traditional alignment techniques (dense or feature-based) may fail. A potential solution is to develop or adapt alignment models that account for differences in modality, resolution, and possibly fine-tune existing models with vegetation scenes (models specialized for sensors and objects of interest).
• Imaging conditions vary between sensors or even between captures from the same sensor (e.g., due to lighting variations, atmospheric effects, etc.), necessitating correction and unification of spectral responses. Since using calibrated references in the field is difficult, self-calibration approaches or cross-calibration between sensors may be explored.
• The differences in spatial and spectral resolution between images require interpolation to achieve optimal resolution. Beyond super-resolution image estimation, the problem can be approached by directly estimating vegetation variables, such as vegetation indices or vegetation cover.
• In practice, real-world data is often noisy and rarely has ground-truth or perfectly clean and aligned data. Self- or semi-supervised approaches can be integrated. Collaboration between denoising, alignment, and fusion can also be considered, rather than treating each module independently.
The goal of the thesis is to develop multi-sensor image fusion tools that address all these challenges. The PhD student will have access to the MIRA platform at GIPSA-lab and its equipment to build controlled datasets tailored to various tasks of interest: alignment, radiometric calibration, denoising, and fusion, using vegetation samples or scenes. The student can build upon the team's previous work which uses deep learning for image fusion or employs hybrid fusion methods combining neural networks and physical models.

Required skills:
* Technical:
- Image processing and computer vision skills: feature detection and matching, image registration and fusion, 3D reconstruction
- Deep learning: CNNs, Transformers
​​- Proficiency in programming languages: Python, C++, CUDA (optional)
- Use of image processing and AI libraries: OpenCV, PyTorch, TensorFlow

* Domain-specific:
- Experience with heterogeneous image fusion
- Development of image processing pipelines
- Conducting literature reviews

* Interpersonal:
- Initiative and ability to work independently
- Ability to work in a team with colleagues from diverse backgrounds
- Communication: writing papers, giving presentations in English

Desired academic background:
Master's degree or Engineering degree specializing in image processing/computer vision and machine learning.

Your Work Environment

Gipsa-lab is a joint research unit of the CNRS (French National Centre for Scientific Research), Grenoble-INP (Grenoble Institute of Technology), and the University of Grenoble, affiliated with Inria (French National Institute for Research in Digital Science and Technology) and the Grenoble Observatory of Earth Sciences.

With 350 people, including approximately 150 doctoral students, Gipsa-lab is a multidisciplinary research unit conducting both fundamental and applied research on signals and complex systems.
Gipsa-lab develops projects in the strategic fields of energy, the environment, communication, intelligent systems, life and health technologies, and language engineering.
Through its research activities, Gipsa-lab maintains a constant link with the economic environment thanks to strong partnerships with businesses.
Gipsa-lab staff are involved in teaching and training at various universities and engineering schools in the Grenoble metropolitan area (Université Grenoble Alpes).
Gipsa-lab is internationally recognized for its research in Automation and System Safety, Data Science (Information and Signal Processing), Speech, and Cognition. The unit conducts its research through 11 teams organized into 4 research departments:
- Automation and System Safety
- Data Science
- Geometry, Learning, Information and Algorithms
- Speech and Cognition
The Gipsa-lab comprises 150 permanent staff and approximately 250 non-permanent staff (doctoral students, post-doctoral researchers, visiting researchers, master's students, etc.)

Constraints and risks

NONE

Compensation and benefits

Compensation

2300 € gross monthly

Annual leave and RTT

44 jours

Remote Working practice and compensation

Pratique et indemnisation du TT

Transport

Prise en charge à 75% du coût et forfait mobilité durable jusqu’à 300€

About the offer

About the CNRS

The CNRS is a major player in fundamental research on a global scale. The CNRS is the only French organization active in all scientific fields. Its unique position as a multi-specialist allows it to bring together different disciplines to address the most important challenges of the contemporary world, in connection with the actors of change.

CNRS

The research professions