Clean up intermediate protocol files

[azazellochg]

This is from I2PC/scipion#2078
This has been requested many times. Personally I don't see a general implementation (JM had some ideas, I'm keen to hear them). As most requests come from relion users, I could make a script to do gentle/harsh clean as done by relion gui and put it into pyworkflow/project/scripts

@pconesa what do you think?

[pconesa]

I think clean up logic resides exclusively to the protocol itself, unless we complicate pyworkflow to understand output files from temporary ones. I'd not go down this way. Therefore the protocol knows what to clean or not.

I see 2 ways, well 3 now with your suggestion:

1. A script to clean up as @azazellochg suggested
2. Each of the protocols, either at create output step or in a specific step make the clean up
3. Pyworkflow, when a protocol finishes, call something like Protocol.cleanFiles, that is empty but each protocol can implement.

2 and 3 are quite similar, maybe 3 more explicit and both does not require the user to manually do the clean.

Now, this works for me, assuming iteration files are in general, not useful once the protocol finishes. The clean up will happens always.

Additionally we can deactivate the cleanup if SCIPION_DEBUG_NOCLEAN is active..this goes better centralised in option 3.

So, I'd go for 3.

[azazellochg]

The clean up will happens always.

That is exactly the problem I see with any general solution. As a user, I do not want to erase any relion output files by default unless I really need to (space issues, project complete etc.). Removing all intermediate iteration files by default will make all iteration viewers useless or broken. Also, I believe Scipion should keep all the metadata / logs etc of underlying programs since this is one of the Scipion's basic goals.

That's why I'm suggesting a script rather than "by default" solution.

[azazellochg]

Looks like 8720c30 has silently addressed this :)

[pconesa]

It is just a call to an empty method... Each protocol should implement the cleanup method for its case. El dom., 14 may. 2023 14:30, Grigory Sharov ***@***.***> escribió:

…

Closed #193 <#193> as completed. — Reply to this email directly, view it on GitHub <#193 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAF7ZYL6AR3E2GEDI4AEU7DXGDF5BANCNFSM4VNENIIA> . You are receiving this because you were mentioned.Message ID: ***@***.*** com>