write capability in Python API

[mads-bertelsen]

Currently the Python API offers reading of MCPL files, would be great to be able to write as well. This would enable quick conversion to other formats with a Python API.

[tkittel]

Thanks for the request @mads-bertelsen . I agree it would be very nice and useful, and it should be a SMOP (simple matter of programming). It is unfortunately also not a small task due to e.g. direction vector packing and the ability to customise contents of MCPL files. And such a feature should also be efficient and accept e.g. numpy arrays. So I can't guarantee a time-scale for this feature to be implemented unfortunately (but it should come at some point...).

[willend]

@mads-bertelsen what do you imagine to read and convert to MCPL?

• Depending on what you want to do, a workaround could be to use a specialised McStas instrument reading your above format and writing it to MCPL? Not Python of course...

[mads-bertelsen]

@willend In PaNOSC we are required to use openPMD standards for HDF5 files, in my case this extension: https://github.com/DavidSagan/openPMD-standard/blob/EXT_BeamPhysics/EXT_BeamPhysics.md
I could use a McStas file to write a MCPL file, but the C API would probably be easier.

[tkittel]

@willend that could technically work (assuming he can read those files with McStas), but if he anyway has to leave Python, then writing his customised hdf5-to-mcpl converter directly in C or C++ with the official MCPL API will certainly introduce less dependencies, as well as allowing to tweak all available MCPL parameters directly. The suggestion of calling mcpl.h via ctypes will even allow him to stay in Python (but certainly, ctypes can be a bit tricky if you haven't use it before).

[willend]

Sure @tkittel not at all an ideal solution. :-) All I know is that @mads-bertelsen do a lot of things that are centred around McStas anyway, so just wanted to make sure he had thought about the possibility.

And as you know I am always prepared to suggest a good little potential shortcut in the form of a hack... ;-)

[tkittel]

Yeah, but your shortcut just sounds like the long-way round.

[willend]

This happens. I also never miss the good old opportunity of name-dropping McStas wherever it has a vague possibility of being useful or where I can demonstrate versatility. :-)

[tkittel]

Really? I had not noticed ;-)

[yrrepy]

One up, this extra native-capability would be really, really useful

[grzanka]

It seems that such a writer may not be so difficult to write. I've managed to write down a few lines of code which convert binary files generated by the SHIELD-HIT12A particle transport code into MCPL file.

The code is here:
https://github.com/DataMedSci/pymchelper/blob/f828dd07f4e00a81f3ef9ae81b8790e2626e52a2/pymchelper/writers/mcpl.py#L25

It doesn't cover all the options for MCPL, but could be extended in an easy way. I am also not sure how efficiently it would handle large (many GBs) amount of data.

[tkittel]

Very nice @grzanka! Indeed, going back to the actual MCPL spec (from the appendix of the MCPL paper) and outputting the bytes directly from Python is the way I want to go eventually. Only thing is that I want to add all the options for what goes in the file + header data, and I want to support large data sets by accepting blocks of numpy arrays of quantities.

In the meantime, I think the code you linked above is a great example which can allow anyone to write MCPL files directly from python! 👍