By continuing to browse the site, you are agreeing to our use of cookies. (More details)

(H/F) Call for Ph.D. thesis applications for a project entitled “Automatic recognition and generation of Cued Speech using deep learning”

This offer is available in the following languages:
Français - Anglais

Ensure that your candidate profile is correct before applying. Your profile information will be added to the details for each application. In order to increase your visibility on our Careers Portal and allow employers to see your candidate profile, you can upload your CV to our CV library in one click!

Faites connaître cette offre !

General information

Reference : UMR5216-CHRROM-011
Date of publication : Friday, March 13, 2020
Scientific Responsible name : Thomas HUEBER et Denis BEAUTEMPS
Type of Contract : PhD Student contract / Thesis offer
Contract Period : 36 months
Start date of the thesis : 1 September 2020
Proportion of work : Full time
Remuneration : 2 135,00 € gross monthly

Description of the thesis topic

Cued speech (CS) is a gesture-based communication system used by deaf people. It uses a set of specific hand shapes and positions to complement the lip information and make all phonemes of a given spoken language clearly visible. The goal of this PhD project is two folds: 1) designing systems aiming at decoding automatically CS into text, 2) generating automatically realistic videos of a “virtual” CS interpreter from text. These modules are the necessary building block of a dialogue systems for deaf people. The successful candidate will investigate advanced deep learning techniques to model the complex relationships between lips, hand gestures, and text in CS. The proposed work plan is the following: 1) extending existing datasets by recording new CS interpreters in both French and English using video and 3D motion capture technique, 2) investigating sequence-to-sequence mapping techniques (based for example on the transformer architecture and GANs) to decode CS into text, 3) investigated video generation techniques based on Conditional VAE or GAN to synthesize automatically realistic CS gestures from text. An extension to the automatic recognition and generation of Sign language could be considered (in collaboration with the LIMSI lab, Orsay, France).
A secondment will take place at the IVèS company (M12-M13 and M24-M25) and at ULB Université Libre de Bruxelles (M19-20). IVèS has developed a strong expertise in phone platform for deaf people integrating end-users aspects and is very well identified at French national and international levels and especially in Grenoble, Toulouse (IVèS had recently acquired the ELIOZ company), and in Montreal. The expertise of the ULB in language development will enable the ESR to evaluate the different technical solutions in relation with language abilities of children with Hearing Implants.

Requested expertise:
Candidates should have a strong background in one of these fields: computer vision, statistics, machine learning, natural language processing, speech processing - must have very good programming skills (in Python), and should have good verbal and written communication skills in English. While not mandatory, basic knowledges in French will be appreciated since most of our datasets are based on French version of CS (called Langue française Parlée Complétée).

Work Context

Gipsa-lab is a joint research unit of the CNRS, Grenoble-INP and the University of Grenoble Alpes; it is an agreement with Inria and the Observatory of Sciences of the Universe of Grenoble.

With a staff of 350 people including about 150 doctoral students, Gipsa-lab is a multidisciplinary laboratory developing fundamental and final research on complex signals and systems. He is internationally recognized for his research in Automatic, Signal and Images, Speech and Cognition and develops projects in the strategic areas of energy, environment, communication, intelligent systems, life and health and linguistic engineering.

Due to the nature of its research, Gipsa-lab maintains a constant link with the business community through a strong industrial partnership.

Its potential as teacher-researchers and researchers is invested in training at the universities and engineering schools of Grenoble (Grenoble Alpes University).

Gipsa-lab develops its research through 12 teams organized in 3 departments:


Images signal


It has 150 permanent and about 250 non-permanent (doctoral student, postdoctoral fellow, visiting researcher, Master student intern ...)

Research will be conducted at GIPSA-lab in Grenoble, France in the CRISSP research team (Cognitive Robotics, Interactive Systems and Speech Processing,

Constraints and risks

Eligibility criteria:
Candidates must have obtained a degree which formally entitles them to embark on a doctorate, either in the country in which the degree was obtained or in the country in which the researcher is recruited. The candidate must have resided or carried out their main activity (work or studies) in a different country from the host organization for at least 12 months in the last 3 years immediately before the recruitment date. Holidays are not counted. Candidates cannot have been awarded a doctoral degree and/or completed more than four years of full-time equivalent research experience.

We talk about it on Twitter!