This repository has been archived by the owner on Apr 26, 2023. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 47
Clean-up feature for Scipion projects #2078
Comments
Thanks Juga. We clearly need to do something. What I'm not sure is if it's
more hassle for the developers to flag all possible output files or just
delete those that are not final, like all iteration files in iterative
protocols.
El mié., 6 may. 2020 16:39, Juha Huiskonen <[email protected]>
escribió:
… The file number in large projects can become hard to manage. It would make
sense to have a feature to delete those files that are not registered as
outputs required by Scipion and that are not generally needed.
Each protocol could define files:
- that are necessary for Scipion workflow (will never be deleted)
- files that are useful in some cases outside of Scipion but not used
in the workflow (deleted only if 'full' clean is selected, e.g. those
diagnostic files that can't be visualised by Scipion)
- intermediate files that are very rarely needed (deleted if 'quick'
or 'full' clean is selected, e.g. iterations 2...N-1)
The user could then select all the protocols they want to clean and select
either of the two options (quick or full).
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#2078>, or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAF7ZYIL2ATS644MQDEOSF3RQFZCXANCNFSM4M2P7XPA>
.
|
The protocols that increase the file count the most seem to be RELION's particle polishing and CTF refinement (especially in projects with many many movies - as these protocols create several diagnostic files for each movie). Refinement and classification runs produces significantly fewer files. |
@azazellochg , @delarosatrevin , It might be a good time now maybe to review the amount of files relion keeps. If you have the chance, rethink if files kept are really useful for a regular user. I'm mention here @dmaluenda to be aware of this for xmipp methods. |
Yes, I plan to have a look at it...but maybe it would be good time to
define one cleanUp method in base Protocol class...and make then actionable
from the GUI. So it is ready for when each protocol decides to implement
it. I have some ideas for a general implementation. We could discuss about
it.
…On Thu, May 7, 2020 at 8:21 PM Pablo Conesa ***@***.***> wrote:
@azazellochg <https://github.com/azazellochg> , @delarosatrevin
<https://github.com/delarosatrevin> , It might be a good time now maybe
to review the amount of files relion keeps. If you have the chance, rethink
if files kept are really useful for a regular user.
I'm mention here @dmaluenda <https://github.com/dmaluenda> to be aware of
this for xmipp methods.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#2078 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/ACAM6WG3OMFHCKNUFMHECQTRQL33HANCNFSM4M2P7XPA>
.
|
Anything that will make life easy for developers is welcome.
…On 7/5/20 20:57, Jose Miguel de la Rosa Trevin wrote:
Yes, I plan to have a look at it...but maybe it would be good time to
define one cleanUp method in base Protocol class...and make then
actionable
from the GUI. So it is ready for when each protocol decides to implement
it. I have some ideas for a general implementation. We could discuss about
it.
On Thu, May 7, 2020 at 8:21 PM Pablo Conesa ***@***.***>
wrote:
> @azazellochg <https://github.com/azazellochg> , @delarosatrevin
> <https://github.com/delarosatrevin> , It might be a good time now maybe
> to review the amount of files relion keeps. If you have the chance,
rethink
> if files kept are really useful for a regular user.
>
> I'm mention here @dmaluenda <https://github.com/dmaluenda> to be
aware of
> this for xmipp methods.
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> <#2078 (comment)>, or
> unsubscribe
>
<https://github.com/notifications/unsubscribe-auth/ACAM6WG3OMFHCKNUFMHECQTRQL33HANCNFSM4M2P7XPA>
> .
>
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#2078 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAF7ZYLWRT7JYYNUJNRBUXDRQMABJANCNFSM4M2P7XPA>.
|
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
The file number in large projects can become hard to manage. It would make sense to have a feature to delete those files that are not registered as outputs required by Scipion and that are not generally needed.
Each protocol could define files:
The user could then select all the protocols they want to clean and select either of the two options (quick or full).
The text was updated successfully, but these errors were encountered: