PhD Position : Safe Manipulation of Deformable Objects with Dexterous Grippers Assisted by Vision (M/F)

Job Description

Thesis Subject

At the CNRS, on the Futuroscope site, the PPRIME Institute, in collaboration with Inria at Rennes, as part of a CNRS research project, O2R and particularly by Action Structuring 2, is recruiting a PhD student to study the safe and adaptive manipulation of deformable objects using multi-fingered grippers operating at different scales, relying on a multi-modal perception combining vision and force measurements. The goal is to develop, based on the TIRREX XXL and TIRREX TRIAGo platforms, a unified grasping and manipulation strategy applicable in both single-arm and multi-arm configurations, enabling the grasping, transferring, and moving of soft objects. This strategy will rely on the dynamic estimation of the deformation state of the deformable object being grasped to adapt in real-time the behavior and control of the grippers.
The work will also include co-manipulation scenarios involving either multiple robots or a robot and a human, to analyze possible synergies and necessary adaptation mechanisms. Finally, the quality of the human-robot interaction will be evaluated through user-centered methodologies to optimize ergonomics, transparency of assistance, and perceived efficiency of shared manipulation.

1-THESIS TOPIC:
-------------------------
The challenges of this thesis focus on the safe and adaptive manipulation of deformable objects using multi-fingered grippers operating at different scales, relying on a multi-modal perception combining vision and force measurements. The goal is to develop, based on the TIRREX XXL and TIRREX TRIAGo platforms, a unified grasping and manipulation strategy applicable in both single-arm and multi-arm configurations, enabling the grasping, transferring, and moving of soft objects. This strategy will rely on the dynamic estimation of the deformation state of the deformable object being grasped to adapt in real-time the behavior and control of the grippers.
The work will also include co-manipulation scenarios involving either multiple robots or a robot and a human, to analyze possible synergies and necessary adaptation mechanisms. Finally, the quality of the human-robot interaction will be evaluated through user-centered methodologies to optimize ergonomics, transparency of assistance, and perceived efficiency of shared manipulation.

2- PROJECT & SCIENTIFIC CONTEXT:
-------------------------------------------------------------
The manipulation of rigid objects has been extensively developed in the literature and is partly resolved when the objects are clearly defined and identified (Seguin et al., 2023). Robotic manipulation of deformable objects—tissues, soft materials, leaves, etc.—remains a central research domain due to the complexity of the object's state (varied shapes, continuous changes, numerous degrees of freedom) and non-linear interaction dynamics. Classical approaches, based on rigid models or static planning, show their limits as soon as the object undergoes significant deformations or there is visual occlusion.
Recently, several studies have made significant contributions. The Foldsformer framework (Mo et al., 2023) exploits a spatio-temporal attention mechanism to plan textile manipulations by sequentially decomposing the manipulation, successfully generalizing to varied fabric shapes (T-shirt, shorts) after only a general demonstration. (Nicola et al., 2024) show that it is possible to estimate the deformation state of a soft material from a depth image via a convolutional network (DenseNet-121) and use this estimation to control the robot in real-time. These approaches illustrate the shift from explicit modeling of soft materials to data-driven perception, more flexible and robust against the variability of shapes.
Other works explore the adaptation of control and the exploitation of the dynamic behavior of the material: model predictive control (MPC) approaches (Qi et al., 2024) or adaptive control (Zimmermann et al., 2021) allow anticipating future deformations and adjusting the robot's commands accordingly—a promising direction to ensure stability and reactivity in complex tasks. Moreover, (Ha & Song, 2022) demonstrate that dynamic actions (catch – stretch - throw) with two robotic arms allow quickly unfolding even large fabrics, extending the robot's physical reach and highlighting the interest of dynamic gestures for deformable manipulation. The work developed in (Deng et al., 2024) links visual perception, instructions expressed in natural language, and strategies for manipulating deformable materials. Robots can thus execute different tasks (fold, pull, reposition) from unambiguous human instructions. These researches pave the way for the manipulation of soft objects by interactive systems, user-centered, capable of understanding high-level objectives.
Recently, the Rainbow team of Inria (Rennes) proposed a fast shape visual servoing approach using the feedback from an RGB-D camera and based on a finite element method (FEM) elastic physical model whose parameters are roughly approximated (Ouafo Fonkoua et al., 2024). This method allows autonomously driving target points on the surface of a deformable volumetric object to desired 3D positions in less than 4 seconds as it has the advantage of not relying on the hypothesis of quasi-static equilibrium of the object's deformation, which is generally considered in the state of the art.
Despite these advances, several challenges remain open: reliable estimation of the complete state of the material in the presence of occlusions, generalization between varied materials, human-robot co-manipulation in dynamic and unpredictable contexts, as well as the integration of adaptive multi-fingered robotic hands, capable of modulating, adapting the grip according to the deformation. To meet these challenges, future systems will need to combine multi-sensor perception (vision, depth, force, contact), models, and adaptive controllers

3- OBJECTIVES & METHODOLOGY:
-------------------------------------------------------------
In this thesis, we propose to exploit the use of multi-fingered robotic grippers at different scales to develop a grasping and interaction strategy capable of apprehending the deformation of the manipulated material based on multi-modal perception. The proposed
approach primarily relies on vision and the evaluation of interaction forces, used both to estimate the state of deformation, detect relevant contact areas, and continuously adjust the manipulation dynamics. The goal is to enable autonomous, flexible, and reliable interactions with a wide variety of soft objects, integrating perception, morphological adaptation, and active control within a single robotic control framework dedicated to grasping and dexterous co-manipulation.
The thesis will focus more particularly on the following aspects:
• Study of methods for characterizing, from measurements acquired by the robotic system (position, force, vision), the mechanical behavior of the soft object under the action of the multi-fingered robotic hands.
• Development of a method for the autonomous grasping of objects with optimization of robot/object contact points to enable the robot to dynamically select the most effective grasping locations that maximize maneuverability for the task of shaping the soft object.
• Development of a multi-modal control strategy based on RGB-D data (color and depth) and contact force measurements provided by the multi-fingered robotic hands.
• Experimental validation of the developments on the TIRREX XXL and TIRREX TRIAGo platforms.


4- SKILLS:
-----------------------------------
The candidate must have an excellent track of records and a Master Degree (or equivalent) in robotics and computer vision AND must have the following qualifications:
• Strong background in robotics
• Experience with computer vision, physical robots, or 3D simulation
• Excellent programming skills in C++
• Excellent written and oral English
• Ability to perform experimental validations
• Ability to work independently as well as collaboratively

Your Work Environment

The thesis will be funded by the PEPR O2R and particularly by Action Structuring 2. It will be co-supervised by researchers from PPRIME-CNRS (Poitiers, France) and Inria (Rennes, France).

The position is full-time for 3 years and will be divided into two 18-month periods:
-The first period, the PhD student will be based in the Rainbow team of Inria at Rennes, France.
-The second period the PhD student will be based in the RoBioSS team at PPRIME-CNRS at Futuroscope, France.

Constraints and risks

Short travels are to be expected, both in France and abroad.

Compensation and benefits

Compensation

2300 € gross monthly

Annual leave and RTT

44 jours

Remote Working practice and compensation

Pratique et indemnisation du TT

Transport

Prise en charge à 75% du coût et forfait mobilité durable jusqu’à 300€

About the offer

About the CNRS

The CNRS is a major player in fundamental research on a global scale. The CNRS is the only French organization active in all scientific fields. Its unique position as a multi-specialist allows it to bring together different disciplines to address the most important challenges of the contemporary world, in connection with the actors of change.

CNRS

The research professions