v4.1: Elmer, M3DC1 fusion-io, docker clones, docker staged builds

[tloobyCFS]

This PR is not only to merge the v4.1 branch into main, but also corresponds to a new docker container release, v4.1. The notable changes in this PR are:

• Replaced handcrafted docker container builds with clones of repos. This means all the code that needs to be compiled in the HEAT environment is now directly cloned and compiled inside the container during build. No more copying files from a development machine into container. This enables other users to build HEAT containers, allows for building containers in CI, and removes dependency on a single development machine for docker container builds. Thanks to @cdrojasm for the idea on this one.
• Now that we are cloning and building inside the container, these builds can be very time consuming. So I have broken the container build into two stages. The first stage is a heatbuilder stage, that compiles all the code. The second stage recycles the heatbuilder stage and then just copies the bins / libs. This saves hours of compile time. Also @cdrojasm's idea.
• M3DC1 fusion-io libs are now directly compiled into this container. I have checked with the D3D integration test.
• Elmer FEM compatibility is now supported in HEAT. This enables the full suite of Elmer FEM models to be employed in HEAT. CFS has additionally developed some UDFs that can be plugged into HEAT for an Elmer metal recrystallization kinetics model.
• A few other patches and feature requests

[tloobyCFS]

noticed bug in GUI mode with new logfiles. need to patch. only effects GUI mode (not TUI)

[tloobyCFS]

@wingena you should download the v4.1 container, and download the v4.1 branch, and see if you can run your 3d plasmas. once you have verified everything is working as intended, approve this PR or comment with the error. thanks.

[wingena]

@plasmapotential I have three comments:

1. Please update EFIT sources in the container.
   The EFIT tools had a recent update moving a function to a faster algorithm. In the function strikeLines() a few calls to the old function remained. This now produces a Deprecation Warning screen output. The functionality is unchanged, but I pushed a fix to the EFIT repo that updates strikeLines() and thereby removes the Deprecation Warning. Please update EFIT sources in the container.

2. The code in the version 4.1 container runs 3 times longer (D3D testcase: 300s with my MAFOT vs 900s with Container MAFOT). The output is the same, it is just much slower. I mounted my MAFOT into the 4.1 container and recovered the original 300s. Maybe the Container compiler is not optimizing some part of the code (Blitz, MAFOT or M3DC1), or because you do not use dynamic libs. I don't know, but a factor of 3 slower for no reason is quite bad.

3. Delete lines 2026 and 2027 in engineClass.py
   

   HEAT/source/engineClass.py

   Line 2027 in 03a6e91

   #shutil.copyfile(self.logFile, self.MHD.shotPath+'HEATlog.txt') #AW: this is a strange place for this command, runHEAT is not complete yet. The same call is already in terminalUI, just after runHEAT is complete

Otherwise, looks okay and passes the test.

[wingena]

One more thing:
Every line in the HEATlog file now has a prefix with a time stamp and the word INFO, like
2024-03-26 22:26:36,552 - INFO -
Do you want this? I think it only makes the log file harder to read.

[tloobyCFS]

@Wingen thanks for the suggestions i did the following:

• removed the lines in engineClass.py
• changed the log output to be only messages (not timestamps or thread ids)
• rebuilt the container, which should have cloned the recent changes to EFIT class so long as they were pushed to github

---

The speed during plasma3D is quite strange. Can you review the MAFOT build process I am using in the Dockerfile and buildMAFOT bash script to make sure everything looks ok? how many nodes / cores are you running this on (so i can scale your times up/down to my machine)?

Dockerfile line that builds MAFOT:

HEAT/docker/Dockerfile

Line 86 in 03a6e91

#clone and build MAFOT

buildMAFOT bash script that is called inside container to build:
https://github.com/plasmapotential/HEAT/blob/03a6e91e7fa98d3e2cebca0efe0cc64531eb45c0/docker/buildMAFOT

---

A couple of things i need from your side:

• review my PR on the MAFOT repo for a container compatible make.inc file
• do any of the plasma3D integration test files (CAD, M3DC1, equilibria) need to go through a GA IP review process before being released to the public domain (in the container)? If yes, maybe best to just leave them out of the container. If no, then i will put them inside the container. just want to avoid any legal implications of uploading GA ip without consent.

[tloobyCFS]

reviewed all files

[plasmapotential]

Reviewed and approved. Removed D3D files until we learn if they are IP approved.