Missions

Adaptive feedback to obtain the illusion of being touched by a virtual agent in an immersive virtual environment
Two years post-doctorate
ANR project
Co-directed by Indira Thouvenin and Catherine Pelachaud

Activités

Research into the definition and classification of social touch (Hertenstein et al., 2006; Kirsch et al., 2018) is a key source of information for specifying the requirements of technological systems to produce credible social touch. These studies show how different types of touch can be defined in terms of their physical properties and how each of them may be more or less suitable for expressing specific emotions and conveying intentions. These studies also show how touch is in itself a truly multimodal sense, with characteristics as diverse as pressure, speed of impact, speed of movement of the touch on the skin (in the case of caressing, for example) or the total duration of the gesture. However, there is currently no tactile interface for expressing the feelings transmitted by an agent to a human being, with a wide range of possibilities (Boucaud et al., 2019). Our hypothesis is that it is possible to augment, and at best induce, the feeling of being touched by other sensory modalities, agency and social finances. The main bottleneck is finding the appropriate combinations of cross-modal audio-tactile correspondences, visual representation of affected body parts, user agency and contextual cues.
In order to study how people can feel affected by a virtual agent in an immersive virtual environment, we will draw on experimental studies carried out on humans, based on fundamental research in cognitive science. The results of these studies will then be integrated into models for simulating tactile movements by virtual agents and for designing 3D adaptive feedback for the immersive virtual environment.

Adaptive feedback in the form of embodied metaphors
Sensory feedback in 3D interactions or 3D metaphors (Bowman et al, 2012) are used in virtual environments (VEs), to help users select or manipulate virtual objects. For example, the ray casting visual metaphor is a virtual ray used as feedback to represent pointing actions on virtual objects when selecting/moving them.
As a general rule, sensory feedback does not provide sufficient variability and must be adapted to the situation and tailored to each user. In the field of training through VE (Jeanne et al., 2017) and in the absence of an instructor during gesture training, learners need enhanced feedback (also called extrinsic feedback). This is information that cannot be given without the help of an instructor or presenter (Sigrist et al., 2013) and is used to help learners visualise gestures, guide them, correct or point out errors.
We want to extend this concept to communication between the agent and the user, by enriching the user's body with adaptive visual and audio comments, also known as "embodied metaphors" (Antle et al., 2009). These augmentations will be adapted to the situation, the social relationship between the agent and the human, and the agent's emotions. For all these reasons, this project aims to create multisensory (sound, visual) embodied metaphors also called "adaptive feedback" (Jeanne et al., 2017) that promote the experience of being touched. The agent's current decision model will be extended in order to choose the appropriate adaptive feedback.

Compétences

The candidate must have a doctoral thesis in computer science, virtual reality, haptics and 3D interaction. Candidate profile and application requirements
The candidate must have a doctoral thesis in computer science, virtual reality, haptics and 3D interaction.

Contexte de travail

The candidate will benefit from the past work of the two supervisors
and some of their students on similar subjects, as well as the general advantages
offered by the Sorbonne in terms of visibility, possibility of funding travel,
etc. This postdoctoral position is linked to the ANR MATCH project, whose members are from the Heudiasyc and ISIR laboratories.

The post is located in a sector covered by the protection of scientific and technical potential (PPST), and therefore requires, in accordance with the regulations, that your arrival be authorised by the competent authority of the MESR.

Le poste se situe dans un secteur relevant de la protection du potentiel scientifique et technique (PPST), et nécessite donc, conformément à la réglementation, que votre arrivée soit autorisée par l'autorité compétente du MESR.

Contraintes et risques

Two sites are planned for this post doc: Compiegne, UTC and Paris UPMC.