PHD : REPRODUCING SCIENTIFIC RESULTS IN HELIOPHYSICS WITH LANGUAGE MODELS (M/F)

Job Description

Thesis Subject

Heliophysics (HP) is a subfield of astrophysics that focuses on the Sun, its interactions with planets and their magnetospheres, and the broader topic of Space Weather. Today, most research in HP is data-driven, with data processing pipelines that produce time series, images, spectra, etc. from observations collected by one or more instruments, or from numerical experiments. The HP community relies on data centers (such as the CDPP, Centre de Données de la Physique des Plasmas) that use standardized metadata and data formats, facilitating access to and reproduction of HP studies. While the topic of scientific paper reproducibility is an active research field, this thesis investigates the reproducibility of HP data processing pipelines using large language models (LLMs) to replicate published computational results.

According to Starace et al. (2025), the state of the art in AI pipelines reproduction by LLMs achieved a 21.0% reproduction rate using Claude 3.5 Sonnet. The generative capacity of LLMs, especially ChatGPT, is currently being discussed within the astronomy community. A recent initiative, PyHC-chat, attempted to adapt GPT models by providing them with relevant code libraries and data endpoints from the PyHC framework to assist heliophysicists in writing code. However, this tool is not designed to reproduce full experimental workflows described in scientific papers; it necessitates some efforts to tailor it to our application.

When being asked about reproducibility in heliophysics data processing experiments, three main research questions arise:
1. In a given scientific paper, which terms, visuals, or cited resources are essential for reproducing the original computational pipeline, and how can they be effectively extracted?
2. How can a model transform these extracted elements into executable code that produces results comparable to those reported in the original study?
3. How can the success rate of reproducibility achieved by LLMs be evaluated and meaningfully interpreted?

A. Data extraction
This thesis draws on BibHelioTech (developed by the CDPP team), which extracts metadata from papers using various rule-based techniques that are insufficient for reproducing scientific workflows, as they rely on predefined patterns and fail to capture contextual information such as data processing descriptions or mathematical formulas. In contrast, combining LLMs with computer vision provides a more flexible framework for understanding and reproducing the computational pipelines described in a paper.

B. Code generation
The reproduction of data processing workflows will rely on a GPT model capable of generating executable code. It will build upon techniques similar to those employed in the aforementioned PyHC-chat, providing the generative model with HP libraries and data endpoints, while also considering fine-tuning of language models.

C. Evaluation
This phase involves evaluating the LLM-generated code by comparing its output plots with those presented in the original paper. Other reproducibility indicators will be very informative when interpreting the LLM's reproduction rate.

Scientific impact
This thesis aims to raise awareness about pipeline reproducibility within the HP community and beyond. A survey will be conducted to investigate researchers' practices when reproducing experiments from published papers and their consideration of reproducibility during the writing process. This work will also result in increasing the quality of data HP repository (like the CDPP and the other associated repositories), with the possibility to test datasets against publications. The CDPP user committee will be involved in assessing the added value of this work.

Environmental impact
Given the high computational and energy costs of LLMs, this work will focus on smaller, domain-specific models.

Your Work Environment

The thesis will be conducted in two research laboratories: LIRA in Meudon for the heliophysics component, and LISN in Orsay for the computer science and natural language processing component.

Status meetings every two weeks, alternately at LIRA in Meudon and at LISN in Orsay.

The PhD thesis will be co-supervised by two advisors: Baptiste Cecconi, Astronomer at the Paris Observatory within LIRA (Heliophysics and Astrophysical Plasmas unit) in Meudon; and Cyril Grouin, Research Engineer (CNRS) at LISN (Semantics and Information Extraction group) in Orsay. B. Cecconi will contribute disciplinary expertise in heliophysics and manages a dedicated data repository for radio astronomical observations in this field. C. Grouin will cover computer science aspects and automated language processing.

We will also collaborate with Vincent Génot, former director of the Plasma Physics Data Center at IRAP in Toulouse, who developed a tool called BibHélioTech, which serves a precursor to this thesis's research. International collaborations are also possible, particularly with the NASA/ADS (Astrophysics Data System) team and within astrophysical data alliances, such as the “Semantics” and “Data Curation and Preservation” groups of the International Virtual Observatory Alliance, or the “Semantics” group of the International Heliophysics Data Environment Alliance.


Computational work can be carried out on the LabIA platform at LISN, the computing facilities of the Paris Observatory, and the Jean Zay supercomputer.
Travel within France and abroad to attend scientific conferences in the field and present research results should also be anticipated.

Constraints and risks

No identified risk

Compensation and benefits

Compensation

2300 € gross monthly

Annual leave and RTT

44 jours

Remote Working practice and compensation

Pratique et indemnisation du TT

Transport

Prise en charge à 75% du coût et forfait mobilité durable jusqu’à 300€

About the offer

About the CNRS

The CNRS is a major player in fundamental research on a global scale. The CNRS is the only French organization active in all scientific fields. Its unique position as a multi-specialist allows it to bring together different disciplines to address the most important challenges of the contemporary world, in connection with the actors of change.

CNRS

The research professions