Contract researcher (M/F): Evaluation of large language models in health

Job Description

Missions

As part of the BPI-funded collaborative project PARTAGES, the mission of this position is to design and coordinate the evaluation of large language models in healthcare and the application of these models to seven healthcare use cases. This includes defining evaluation protocols for each use case according to the type of case and its application framework; implementing specific protocols for evaluating use cases and foundation models; and monitoring the results of algorithm evaluations for each use case and foundation model. The latter tasks will involve close interaction with the partners responsible for developing the language models and the partners responsible for use cases, as well as organizing evaluations similar to evaluation campaigns and feedback workshops. Methods and results will be documented and published in relevant conferences and journals.

Activity

In the context of the PARTAGES project's evaluation of natural language processing (NLP) systems, mainly based on neural methods, which will be applied to the medical field:
- designing and implementing state-of-the-art methods for the evaluation of NLP systems,
- managing the evaluation activities of project partners,
- planning and running project meetings,
- participating in project meetings,
- creating and implementing evaluation protocols,
- analyzing evaluation results,
- drafting reports, guides, and publications,
- attending scientific conferences.

Your Profil

Skills

- Thesis in natural language processing with machine learning,
- mastery of NLP and machine learning methods and tools, including neural learning,
- mastery of NLP evaluation measures and protocols,
- experience of collaborative research projects,
- taste for teamwork,
- easy contact with external teams,
- time management and punctuality in deliverables,
- fluency in French and technical English.

Your Work Environment

LISN Laboratory
https://www.lisn.upsaclay.fr/
The Interdisciplinary Laboratory for Digital Sciences (LISN) is a Joint Research Unit (UMR 9015) created on January 1, 2021, through the cooperation of the LIMSI and LRI research teams. Its supervisory bodies are CNRS, Université Paris-Saclay, INRIA, and CentraleSupélec. With a staff of almost 400, the institute's research areas span the core themes of digital and engineering sciences, as well as interdisciplinary topics such as artificial intelligence and data science, human-computer interaction, natural language and speech processing, and bioinformatics.

Research activities are organized into 5 departments, 5 cross-disciplinary actions, and 16 research teams. 5 research support teams ensure the smooth running of the laboratory. The work will take place within the SEME team of the Language Sciences and Technologies department.

The PARTAGES project
https://www.health-data-hub.fr/actualites/democratiser-lia-generative-en-sante-letat-travers-france-2030-selectionne-le-projet
The PARTAGES project, led by a consortium of some thirty partners including research laboratories (from CNRS, INRIA, and various universities), healthcare facilities, and deep-tech companies, is one of the winners of the France 2030 call for projects on generative AI. Coordinated by the Health Data Hub (HDH), this project aims to use generative artificial intelligence to help healthcare professionals save precious time in clinical practice.

LISN's SEME team is responsible for the project's Evaluation work package, which will be the new recruit's main focus. The recruit will be working in particular with Pierre Zweigenbaum and Aurélie Névéol, as well as with another contract researcher already recruited for the project. Collaboration will also take place with LISN's ASARD team, which is responsible for the project's evaluation infrastructure, under the direction of Anne-Catherine Letournel.

The position is located at LISN in Orsay. It will involve travel to partner sites, meetings, and conferences.

Constraints and risks

working in front of a screen

Compensation and benefits

Compensation

Between 3131 and 4341,70 € monthly gross salary

Annual leave and RTT

44 jours

Remote Working practice and compensation

Pratique et indemnisation du TT

Transport

Prise en charge à 75% du coût et forfait mobilité durable jusqu’à 300€

About the offer

About the CNRS

The CNRS is a major player in fundamental research on a global scale. The CNRS is the only French organization active in all scientific fields. Its unique position as a multi-specialist allows it to bring together different disciplines to address the most important challenges of the contemporary world, in connection with the actors of change.

CNRS

The research professions