Missions

As part of the BPI-funded collaborative project PARTAGES, the mission of this position is to design and coordinate the evaluation of large language models in healthcare and the application of these models to seven healthcare use cases. This includes defining the constituent elements of evaluation protocols, defining model comparison dimensions, setting up an evaluation protocol for foundation models, defining evaluation protocols for each use case according to the type of case and its application framework, implementing specific protocols for the evaluation of use cases and foundation models, and monitoring the results of evaluations of algorithms for each use case and foundation model. The latter tasks will involve close interaction with the partners responsible for the development of the language models and the partners responsible for the use cases, the organization of evaluations similar to evaluation campaigns, and the organization of feedback workshops. Methods and results will be documented and published in relevant conferences and journals.

Activities

In the context of the PARTAGES project's evaluation of natural language processing (NLP) systems, mainly based on neural methods, which will be applied to the medical field:
- design and implementation of state-of-the-art methods for the evaluation of NLP systems,
- managing the evaluation activities of project partners,
- planning and running meetings,
- participating in meetings,
- creation and implementation of evaluation protocols,
- analysing evaluation results,
- drafting reports, guides and publications.

Skills

- Thesis in natural language processing with machine learning,
- mastery of NLP and machine learning methods and tools, including neural learning,
- mastery of NLP evaluation measures and protocols,
- experience of collaborative research projects,
- taste for teamwork,
- easy contact with external teams,
- time management and punctuality in deliverables,
- fluency in French and technical English.

Work Context

LISN Laboratory
https://www.lisn.upsaclay.fr/
The Interdisciplinary Laboratory for Digital Sciences (LISN) is a Joint Research Unit (UMR 9015) created on January 1, 2021 through the cooperation of the LIMSI and LRI research teams. Its supervisory bodies are CNRS, Université Paris-Saclay, INRIA and CentraleSupélec. With a staff of almost 400, its research areas cover the core themes of digital and engineering sciences, as well as interdisciplinary topics such as artificial intelligence and data science, human-computer interaction, natural language and speech processing, and bioinformatics.

Research activities are organized into 5 departments, 5 cross-disciplinary actions and 16 research teams. 5 research support teams ensure the smooth running of the laboratory. The work will take place within the SEME team of the Language Sciences and Technologies department.

The PARTAGES project
https://www.health-data-hub.fr/actualites/democratiser-lia-generative-en-sante-letat-travers-france-2030-selectionne-le-projet
The PARTAGES project, led by a consortium of some thirty partners including research laboratories (from CNRS, INRIA and various universities), healthcare facilities and deep-tech companies, is one of the winners of the France 2030 call for projects on generative AI. Coordinated by the Health Data Hub (HDH), this project aims to use generative artificial intelligence to help healthcare professionals save precious medical time.

LISN's SEME team is in charge of the project's Evaluation work package, which will be the main focus of the new recruit's work. The recruit will be working in particular with Pierre Zweigenbaum and Aurélie Névéol. Collaboration will also take place with LISN's ASARD team, in charge of the project's evaluation infrastructure, under the responsibility of Anne-Catherine Letournel.

The position is located at LISN, in Orsay. It will involve travel to partner sites, meetings and conferences.

The position is located in a sector under the protection of scientific and technical potential (PPST), and therefore requires, in accordance with the regulations, that your arrival is authorized by the competent authority of the MESR.

Constraints and risks

working in front of a screen