This repository shows two different model strategies adapted from Transformers [1] and Packed-Ensembles [2] for solving the RANS equations, based on the LIPS framework [3] and the Airfrans Dataset [4].

The study provided here is part of the ML4physim challenge hosted by IRT-Systemx (see Codabench page). CFD simulations being very costly, the use of data-driven surrogate models can be useful to optimize the shape of airfoils without paying the cost of expensive simulations.

Two families of models were implemented and tested here:

• Subsampled Bi-Transformer models: which are modified version of transformer networks, where for each simulations, the query tokens are only attended to a subsampled set of key tokens from the pointcloud of the simulation which we call the skeleton of the mesh. The best model's implementation is locateed in the subsampled_bi_transformers/bi_transformer folder, and can be ran using the run.py file.
• Packed-ensemble models: Packed Ensembles are generalizations of Deep-ensembles that allow to lower the number of a classical ensemble model's parameters. For Packed-Ensembles, two frameworks are proposed in the packed_ensembles folder:
  • A complete and independent framework developped in ml4science.ipynb with a custom training function and a cross validation selection implementation.
  • An implementation of the Packed-Ensemble model within the LIPS framework in packed_lips.ipynb. All the configurations that were tried are explicited in the config.ini file.

The Bi-transformer model got us the $4^{\text{th}}$ place in this challenge!

Also, feel free to checkout the checkpoint branch to see other model tests and implementations.

conda create --name ml4science python=3.9

conda activate ml4science

Download the LIPS repository in the src folder

cd src
git clone https://github.com/IRT-SystemX/LIPS.git

Then remove the numpy and scipy requirement from the setup.py file to avoid conflicts.

cd LIPS
pip install -U .
cd ..

Checkout https://pytorch.org/get-started/locally/

pip install airfrans

import os
import airfrans as af

directory_name='Dataset'
if not os.path.isdir(directory_name):
    af.dataset.download(root = ".", file_name = directory_name, unzip = True, OpenFOAM = False)

pip install torch-uncertainty

• Anthony Kalaydjian, Master student @ ENSTA/EPFL - [email protected]
• Anton Balykov, Master student @ EPFL - [email protected]
• Adrien Chan-Hon-Tong, Researcher in ML @ Onera Université Paris Saclay – [email protected]

[1] Attention Is All You Need, A. Vaswani et al. (2017).

[2] Packed-Ensembles for Efficient Uncertainty Estimation, O. Laurent et al. (2023).

[3] LIPS - Learning Industrial Physical Simulation benchmark suite, M. Leyli Abadi et al. (2022).

[4] AirfRANS: High Fidelity Computational Fluid Dynamics Dataset for Approximating Reynolds-Averaged Navier-Stokes Solutions, F. Bonnet et al. (2023).