Faites connaître cette offre !
Reference : UPR3251-JULFRA-002
Workplace : ORSAY
Date of publication : Thursday, September 10, 2020
Type of Contract : FTC Scientist
Contract Period : 17 months
Expected date of employment : 1 November 2020
Proportion of work : Full time
Remuneration : 2695€ to 3841€ monthly before tax, according to experience
Desired level of education : PhD
Experience required : 1 to 4 years
Context: ANR Project ELEMENT
Because memorizing and executing gestures is challenging for users, most current approaches to movement-based interaction consider intuitive interfaces and trivial gesture vocabularies. While these facilitate adoption, they also limit users' potential for more complex, expressive and truly embodied interaction. Considering movement-based interaction beyond the mouse-keyboard paradigm, the ANR project ELEMENT (Enabling Learnability in Embodied Movement Interaction) proposes to shift the focus from intuitiveness towards learnability: new interaction paradigms require users to develop specific sensorimotor skills compatible with – and transferable between, – digital interfaces (including video interface, mobile devices, internet of things, game interfaces). With learnable interactions, novice users should be able to approach a new system with a difficulty adapted to their expertise, then the system should be able to carefully adapt to the improving motor skills, and eventually enable complex, expressive and engaging interactions. The long-term aim is to foster innovation in multimodal interaction, from assistive technologies to media interaction in creative applications.
The use of continuous multisensory feedback as a medium to support movement learning is showing promise in a number of applications such as sports, performing arts or rehabilitation [1, 2]. Such motor learning support systems require fine-grained movement modeling to provide appropriate audio, visual or haptic feedback to assist the learner in acquiring or recovering particular motor skills. In this context, there is a need for advanced computational models able to take into account sensorimotor learning mechanisms . Complementary to analytical models of movement learning, the development of data-driven strategies using machine learning is a core focus of the ELEMENT project . Such models have the potential to dynamically adapt to the skill level of the learner (relative to a given task) so as to provide appropriate feedback.
This postdoc project focuses on the design, implementation and evaluation of adaptive multisensory feedback systems to assist movement learning in the context of dance practice. In particular, we will study how non-visual feedback (auditory and/or haptics feedback) can be used to support the learning of complex movement sequences that can be found in dance. By complex movement sequences, we consider sequences of multiple movement units (or gestures) which must be executed with specific properties (e.g. trajectory, dynamics, movement quality) and articulated together. Modeling such sequences requires the development of appropriate analysis techniques that can capture various properties of the movement, such as trajectories, dynamics, or even movement qualities [5, 6], as well as their relationship to movement learning. Methods such as micro-movement analysis  could be used to track the evolution of these movement characteristics over time, as a function of the performance. Moreover, the articulation between the various segments of a complex movement sequence evolves with the expertise of the person. Following preliminary work from the ELEMENT project, we will investigate how to model articulation in such complex movement sequences [8, 9], and how to design appropriate feedback on movement articulation.
The main application context of this postdoctoral project is dance learning, with a focus on non-visual modalities. Applications to movement learning support systems for people with vision impairments will be considered, according to the candidate's background and potential collaborations with local organisations.
The main research questions of the proposed postdoctoral project are:
How do people learn complex dance movement sequences, in particular for people with vision impairments?
What representations of the movement can be combined to capture multiple properties of the movement (trajectories, dynamics, expressivity, articulation, etc.) in a complex sequence?
What computational models can quantify and track these properties as a function of the learning process of a person?
How to design continuous multisensory feedback in order to support the learning of these various properties in a complex movement sequence?
Can adaptive feedback systems improve the learning rate and learning experience?
Conduct a state of the art on the field of adaptive methods in movement modeling, adaptive systems for movement learning and dance, and auditory feedback for movement learning
Design and conduct qualitative studies (interviews, observational workshops) to analyze the existing practices in dance learning, in particular for the case of people with visual impairments.
Using a participatory approach, design and implement one or several interactive systems providing continuous audio feedback for movement learning. The design process will be iterative and will include intermediate evaluation workshops as well as motion capture sessions to constitute databases of movement learning. Knowledge about movement skill acquisition from initial observations will be used to inform the design of the sonification strategies. The design will focus on integrating adaptation mechanisms to adapt the sound feedback to the expertise and learning process of the person.
Design and conduct a controlled experiment to evaluate the proposed system. The study should assess the effectiveness of the feedback system to help acquire the various properties of the movement sequence, in particular regarding the adaptivity to the skill level of the participant.
Publications and dissemination.
We are looking for passionate candidates with creativity and scientific curiosity qualities, strong problem solving skills, good knowledge of signal processing, machine learning and HCI, as well as experimental methodologies and statistical analysis. Candidates are expected to have a strong background in movement and computing, especially regarding movement analysis and processing, as well as design and evaluation of interactive movement-based systems. Experience in creative application contexts such as dance or performing arts, and interest in working on interaction for people with disability are appreciated. Candidates should demonstrate a track record of research publications in the field of movement and computing (MOCO, CHI, DIS, TOCHI, JMUI, ...) and other movement related conferences and journals.
This full time postdoctoral position will take place at LIMSI as part of the ANR ELEMENT project. The ELEMENT project is coordinated by Ircam (Paris), and also involves LRI (Orsay) and LIMSI (Orsay). It is expected that the post-doctoral student will interact strongly with the other doctoral and post-doctoral students of the project, and will work in collaboration with all partners. Travel for experiments or conferences may be funded during the postdoct.
The post-doctoral student will be equipped with a laptop computer and accessories. LIMSI has acquired several portable motion sensors (IMU, EMG sensors), and has a whole-body motion capture system that can be used for data collection, prototyping, and evaluation of the computational models developed. Additional equipment can be acquired during the project.
Constraints and risks
- risks : work with display screens
We talk about it on Twitter!