PhD (M/F) : From Tactile Estimation to Control: Towards Vision/Tactile Shape Manipulation of Flexible Objects under Uncertainty

Job Description

Thesis Subject

Manipulating deformable objects requires reasoning about shape, contact forces, local compliance, friction, and uncertainty. Vision-based shape servoing can reconstruct and control the visible deformation of soft objects, and online FEM can extend this estimate to occluded regions. Nevertheless, perception based on vision alone does not observe the local contact mechanics at the grasp or along the robot body. These hidden conditions often determine the task outcome: a similar visible shape may arise from different stiffness distributions, contact conditions, incipient slip, or force distributions. This thesis develops a tactile-informed, uncertainty-aware framework for dual-arm manipulation of deformable objects, in which a physical belief state provides the interface between model-based estimation, safe control, and generative decision-making. A central scientific question is which physical quantities are identifiable from tactile interactions and how this information can be used for control and policy learning. The thesis therefore targets task-relevant physical beliefs, such as local normal and tangential compliance, effective bending stiffness, damping, friction or slip state, grasp boundary conditions, and uncertainty over the predicted deformation. This belief state is designed to retain the information required for robust deformable manipulation while keeping the estimation problem well-posed and directly exploitable by both control algorithms and learned policies.

The first objective extends a SOFA-based deformable-object digital twin with tactile, force, and proprioceptive feedback. SOFA serves primarily as a real-time forward simulator and structured prior: predicted deformations, forces, and contact states are compared against RGB-D, wrist force/torque, joint torques, and tactile signals. The thesis investigates a hierarchy of estimation methods, including reduced-order sensitivity models, local interaction matrices, derivative-free updates for low-dimensional parameters, learned residual models, and, when appropriate, implicit differentiation through selected reduced solver components. The output is a calibrated physical belief state over object shape, contact state, local compliance, slip risk, and prediction uncertainty.

The second objective characterizes tactile identifiability and active probing. The thesis studies how active tactile exploratory motions, contact location, grasp configuration, object geometry, and sensor modality affect the observability of compliance, and friction, distinguishing beliefs recoverable from passive contact from those requiring active excitation. Because tactile measurements are local, informative estimation requires purposeful motion: the robot will actively select small, safe probing actions that reduce belief uncertainty while respecting force and stress limits, turning the grasp into an estimation instrument.

The third objective uses the resulting belief state and its calibrated uncertainty for tactile/force- aware control. A model-predictive or hybrid force-position controller regulates both deformation and contact forces while accounting for uncertainty: it favors information-gathering actions when the belief is highly uncertain, enforces stress or force constraints with margins that widen as observability degrades, and uses tactile feedback to detect contact state changes (slip, unexpected compliance) and trigger re-estimation. This provides an interpretable and safe control layer for contact-state estimation and manipulation under object deformation.

In collaboration with ClearLab/NUS, the aforementioned aspects will be investigated as a conditioning representation for uncertainty-aware generative policies, such as diffusion policies, for longer-horizon routing, fixture insertion, and collaborative tasks. The policy conditions its action generation on the estimated shape, contact state, compliance, slip margins, and uncertainty provided by the tactile-physical estimation layer. Learning is therefore guided by physical predictions and tactile evidence, while the policy contributes long-horizon decision-making and data-efficient action generation under uncertainty. This provides a natural division of work with ClearLab/NUS. The France-based work will focus on tactile/FEM belief estimation and robust control, while the NUS-side work will focus on belief-conditioned generative policies for deformable-object manipulation.

The expected outcome is a framework for manipulation of deformable objects, combining physical estimation, active probing, robust force/tactile control, and belief-conditioned generative policies. Experiments will progress from controlled deformable beams and foam objects to deformable linear objects such as cables, hoses, and flexible pipes, using the LIRMM dual-arm platform equipped with force/torque sensing, accurate joint torque measurements, and capacitive tactile/proximity skin. Evaluation will measure shape error, force regulation, slip detection, stress-limit violations, uncertainty calibration, data efficiency, and generalization across objects with different stiffness and contact properties.

Your Work Environment

As part of the Embodied AI project involving CNRS laboratories in France and Singapore (www.cnrsatcreate.cnrs.fr/embodied-ai/), this thesis will aim to develop methods for estimating and controlling a robotic arm to manipulate the shape of soft objects by combining vision and touch in situations of uncertainty. Working meetings and ongoing discussions with Harold Soh, a professor at the National University of Singapore (NUS), will take place throughout the doctoral program. A stay at NUS for approximately 6 months during the program is also planned.

Constraints and risks

The person selected for this position (M/F) will work with the National University of Singapore (NUS) to develop tactile estimation and control for a robotic arm designed to manipulate the shape of soft objects, combining vision and touch in uncertain conditions.

Compensation and benefits

Compensation

2300 € gross monthly

Annual leave and RTT

44 jours

Remote Working practice and compensation

Pratique et indemnisation du TT

Transport

Prise en charge à 75% du coût et forfait mobilité durable jusqu’à 300€

About the offer

About the CNRS

The CNRS is a major player in fundamental research on a global scale. The CNRS is the only French organization active in all scientific fields. Its unique position as a multi-specialist allows it to bring together different disciplines to address the most important challenges of the contemporary world, in connection with the actors of change.

CNRS

The research professions