Neurodamus bump and implementing stable dependency

[ferdonline]

Creates version specification which depend on both the model and core versions.

E.g. using model 1.1 and core 3.0.1 it will define a version '1.1-3.0.1' which uses tag 1.1 and depends on core 3.0.1

[ferdonline]

Ohh.. damn pep8! But CI tests pass :)

[matz-e]

With the current implementation, this will generate modules for all neurodamus versions found.

[matz-e]

The next iteration of the deployment would solve this…

[ferdonline]

last commit adds @1.1-3.0.2 but the deployments scripts don't like it

[matz-e]

I'm not surprised...

[ferdonline]

Actually it was due to the double @@ typo :)
In the end, if you don't think is too bizarre I would like to go forward with it, and no worries, you can blame me for any problem, I will do my best at maintaining it. Just le me know if you see a better way at achieving a similar result, or you are happy with the implementation

[matz-e]

It still looks freakish to me, personally. Remove that comment above, and I'll approve, though. Technically nothing wrong with it…

[pramodk]

If I am not mistaken this would be first package in Spack to do dynamic version generation and hence I am hesitant about this. Technically seems ok but is this way we should adopt?

Apart from neurodamus use case, this issue of specific version requirement exist everywhere. I am wondering if we should solve this issue in neurodamus in different way e.g. using resource.

I will let @matz-e to comment on this better...

[ferdonline]

dynamic version generation

We still statically maintain the list of versions, it's not automatic

Apart from neurodamus use case, this issue of specific version requirement exist everywhere. I am wondering if we should solve this issue in neurodamus in different way e.g. using resource.

Don't agree. Neurodamus-model is a special case because it incorporates code from the core (making 90% of the code or so). So actually every time we bumped the core, neurodamus-xxx changed and the version didn't. It has been a long pending issue to have neurodamus-xxx versions being meaningful. I don't think this logic should be used anywhere else.

[pramodk]

Don't agree. Neurodamus-model is a special case because it incorporates code from the core

My comment is in the context of packages like LLVM where you can see what happens:

spack/var/spack/repos/builtin/packages/llvm/package.py

Line 99 in 967c78d

base_url = 'https://llvm.org/releases/%%(version)s/%(pkg)s-%%(version)s.src.tar.xz'

LLVM composed of 10 different components. The question is how you define the version of LLVM if you bump up the version of one of it's resource.

[ferdonline]

As far as I see in LLVM, all those extra components follow the same version.
I don't know of many cases where we actually put two independent codes, supposedly developed by different people, following completely different versions and make a single thing. I also think that knowing each individual version is very helpful for both developer and users.
That was the original intent. This is not an attempt to solve the spack multi-deployments-per-version problem, although it helps.
Anyway, I don't claim this is a perfect solution, and if you believe this would be useful for other packages, please go ahead and improve it to make more generic. I just feel happy with the result given the effort.

[matz-e]

I'm not surprised...

[pramodk]

I don't know of many cases where we actually put two independent codes, supposedly developed by different people, following completely different versions and make a single thing. I also think that knowing each individual version is very helpful for both developer and users.

this kind of done via submodule right? e.g. coreneuron is incomplete without mod2c. So we have mod2c as submodule into coreneuron (and mod2c can be deployed as separate standalone version).

Technically this change looks Ok. Just that more logic into package recipe caught my attention.

By the way, looking at module usage on BB5, I am wondering if / how people are using neurodamus-core.

[ferdonline]

please retest

[ferdonline]

this kind of done via submodule right? e.g. coreneuron is incomplete without mod2c.

@pramodk maybe we can have a call if you want to discuss this and look for solution together. I would say it's still a different case, since both are perfectly independent applications which, only for user convenience, we put together in a pack. Let me elaborate my view better.
Since the split of neurodamus from the models the two things became effectively independent. We can have only the core and we can have only the model, the advantage being that scientists can easily develop and compile their models and integrate with one neurodamus-core.
The way this integration is done can nowadays be twofold: (1) the old way of "compile together" and (2) since January, compile special from model only and load libs, which is what bglib-py does it. (For the record is also supported in neurodamus-py: load neurodamus-core~commonmods, set BGLIBPY_MOD_LIBRARY_PATH and launch)
So, IMO, this is a software pack "neurodamus+model" (hence the name). However we were using the model version to version the "pack", which is IMO bad. Because of that, even when we added new important features to core, core+model didn't see a new version, which probably confused users. We saw that most of them even started relying on the Archives, since the version was meaningful-less: [email protected] from May was a very different thing from [email protected] from October.
I have acknowledged this problem since long but the only solution I saw before was to have Git projects only with two submodules: core and model... but that insanely complicates the setup.
Given this python implementation ended being relatively simple I thought we can use it in this rare case. But if you have ideas let me know, I would be happy to solve the problem in a better way