PhD Position M/F : Visuo-Tactile Perception and Hybrid Control for Robotic Manipulation

Job Description

Thesis Subject

Robots are increasingly expected to leave structured industrial cells and operate in open environments such as farms, workshops, warehouses, hospitals, homes or outdoor intervention sites. Recent progress in legged locomotion has shown that robots can become increasingly capable of moving through complex environments. In parallel, approaches such as [4] suggest that vision-language-action models can connect language-level task descriptions, visual scene understanding and candidate robot actions. However, mobility and high-level action proposals are not sufficient: to be genuinely useful in the field, robots also need dexterity. They must be able to grasp, move, reconfigure and interact with objects whose geometry, physical properties and contact conditions are only partially known.
This PhD project addresses robotic manipulation in less controlled environments, where classical rigid-object manipulation based mainly on 6D pose estimation is only a partial answer. Many objects encountered in practice are deformable, articulated, flexible, partially occluded or affected by contacts during interaction. Their state cannot always be reduced to a rigid pose: shape, local geometry, contact conditions, grasp stability, material response and task-related affordances may all become relevant for action.
Recent reviews on robotic cloud manipulation [2], as well as benchmark efforts on grasp selection such as [5], show that deformable and textile-like objects clearly expose the limitations of pose-based manipulation. The robot must reason about shape evolution, contacts and partial observability rather than only estimating a rigid transformation.
This is where hybridization becomes essential. Model-based approaches bring structure, physical consistency and control-oriented constraints, but they are often incomplete when the robot interacts with objects whose geometry, material properties and contact conditions are only partially known. As discussed in [1], data-driven approaches can learn visual descriptors, latent states, residual dynamics or action priors from experience, but they may require large datasets and do not naturally provide safety or stability guarantees. Real-world manipulation therefore calls for methods that combine both: models to constrain and guide action, and data to adapt to phenomena that are difficult to model explicitly.
The thesis is positioned within the PEPR Robotique, and more specifically within Targeted Project 2 (TP2), dedicated to the hybridization of model-based and data-driven methods for motion generation in robotics. The proposed work will contribute to this PEPR objective through the context of contact-rich manipulation. Recent visuo-tactile affordance representations [3] also illustrate the interest of combining external visual perception with local tactile feedback for contract-rich manipulation. The central idea is to develop hybrid perception-control strategies that combine geometric and physical priors, learned representations, multimodal sensing and safety-aware control, so that the robot can act with both adaptability and physical consistency.
Research plan and expected contributions
The thesis will focus on three main contributions:
• Represent object states beyond 6D pose. Build compact visuo-tactile representations combining RGB-D geometry, local shape, contact cues, tactile signals and learned descriptors.
• Hybridize model-based and data-driven control. Develop closed-loop manipulation strategies where physical/geometric constraints guide learning, and learned components improve adaptation to complex contacts, deformable behavior and uncertainty.
• Ground high-level action priors in physical feedback. Use foundation models or vision-language-action models to propose goals, actions or manipulation primitives, and refine them through visuo-tactile feedback, model-based constraints and safety-aware control.
Experimental work will be carried out on a Franka Research 3 (FR3) equipped with visual and tactile sensing. The expected outcome is a hybrid perception-control framework for real-world robotic manipulation, where the robot adapts its actions from rich object-state representations and real-time physical feedback.

[1] Ai, B., Tian, S., Shi, H., Wang, Y., Pfaff, T., Tan, C., Christensen, H. I., Su, H., Wu, J., & Li, Y. (2025). A review of learning-based dynamics models for robotic manipulation. Science Robotics.
[2] Longhini, A., et al. (2025). Unfolding the Literature: A Review of Robotic Cloth Manipulation. Annual Review of Control, Robotics, and Autonomous Systems, 8, 295–322.
[3] Wu, Q., Wang, H., Zhou, J., Xiong, X., & Lou, Y. L. (2025). TARS: Tactile Affordance in Robot Synesthesia for Dexterous Manipulation. IEEE Robotics and Automation Letters, 10(1), 327–334.
[4] Zhao, W., Chen, J., Meng, Z., Mao, D., Song, R., & Zhang, W. (2024). VLMPC: Vision-Language Model Predictive Control for Robotic Manipulation. Robotics: Science and Systems.
[5] De Gusseme, V.-L., Lips, T., Proesmans, R., …, Yamazaki, K., Mateo-Agulló, C., Verleysen, A., & Wyffels, F. (2026). A Dataset and Benchmark for Robotic Cloth Unfolding Grasp Selection: The ICRA 2024 Cloth Competition. The International Journal of Robotics Research.

Your Work Environment

This thesis will be carried out within the Interdisciplinary Carnot de Bourgogne Laboratory (ICB), a Joint Research Unit of CNRS / UBE / UTBM including more than 350 physicists, chemists, engineers and technicians based in Bourgogne-Franche-Comté, at the sites of Dijon, Le Creusot, Chalon-sur-Saône and Belfort (Sévenans). The ICB develops new functionalities in optics and for the materials of the future, targeting applications in industry (photonics, metallurgy, Industry 4.0, …), medicine, high-speed optical communications, information processing at the nanometric scale, energy and quantum technologies. Recently, the ICB created a new department named AI-Computing and Cyber-physical Systems. It is within this department in Dijon that this thesis offer is situated. It is funded by the PEPR Robotique HAMMER project and will be supervised by 2 ICB members (Carlos Mateo and Cédric Demonceaux) in close collaboration with the I3S laboratory at Université Côte d'Azur (Guillaume Allibert).

Constraints and risks

-

Compensation and benefits

Compensation

2300 € gross monthly

Annual leave and RTT

44 jours

Remote Working practice and compensation

Pratique et indemnisation du TT

Transport

Prise en charge à 75% du coût et forfait mobilité durable jusqu’à 300€

About the offer

About the CNRS

The CNRS is a major player in fundamental research on a global scale. The CNRS is the only French organization active in all scientific fields. Its unique position as a multi-specialist allows it to bring together different disciplines to address the most important challenges of the contemporary world, in connection with the actors of change.

CNRS

The research professions