Packages/Modules and beyond

[orefalo]

Hi again!

I have now built 4 extensions for murex

• a JavaHome helper
• The linux Free command for OSX
• A general command colorizer
• My personal profile

I will soon start working on

• The starship smart prompt

and of course, I looked at some of your own modules, in particular

• Jump! - Love it
• some typescript shortcuts
• docker-compose completion

Allow me to ask a few questions:

• Do you have plans for a central module repository, similar to go or npm, where users can search and install packages?
• Do you have plans to implement 'murex-package uninstall' ? I had to manually delete the modules in .murex*, not trivial at start
• Can packages automatically install other (murex) dependencies ?
• Is it possible to install only a module within a package? For instance, I would like to use the docker-compose completions alone.

Thank you

[lmorg]

Do you have plans for a central module repository, similar to go or npm, where users can search and install packages?

No plans as yet but it's an idea I'm open to. There would need to be a low overhead of administrating and building out such an ecosystem though. Possibly just using git initially like Homebrew does/did.

Do you have plans to implement 'murex-package uninstall' ? I had to manually delete the modules in .murex*, not trivial at start

Weirdly I hadn't even thought about uninstall but yes you're right. murex-package definitely needs that. I can get it added for the next release.

Can packages automatically install other (murex) dependencies ?

No. I did consider automatically resolving dependencies but given dependencies are often going to be external commands (eg docker-compose and installing those dependencies is going to differ wildly from one OS to another (even different workstations on the same OS like people who use homebrew vs mac Ports) it would be hard to get dependency resolution to work, maybe even impossible to get it to work well. And I'm not really sure there is much benefit of having murex modules resolve their murex dependencies if everything else that the module depends on cannot be automatically resolved.

That all said, this is a technical push back rather than something I don't want. So the right technical design would definitely change my mind.

Is it possible to install only a module within a package? For instance, I would like to use the docker-compose completions alone.

At this stage, no. It's a good idea but the only way it could be achieved would be to install the whole package and disable unused modules within. It's definitely a feature I could add though.

---

Nice work on your modules by the way. A couple of notes, if you don't mind:

• on free, you can specify the OS in the modules.json. eg "Platform": [ "darwin" ]. Currently this will produce an error when trying to load it on non-Darwin platforms though (maybe that's not the best behaviour?) so there's no harm in what you've done either.
• where you've put if { $SELF.Interactive } then { that's already covered because murex packages are not loaded if the shell isn't in interactive mode (intentional behaviour so that shell scripts can be consistent in different environments). You can force shell scripts (and -c ... commands too) to load modules via -load-modules flag, eg murex -load-modules.

---

The free module looks really useful though. Mind if I pull it too?

[orefalo]

Thank you for all the suggestions,

• You are welcome to pull free - I have yet to improve formatting with sprintf or printf
• with your permission and based on the discussion above, I will create independent versions of -compose & typescript - modules
• managing module/package dependencies can be challenging - no need to overengineer at this point - KISS principle
• So modules are interactive by default, will check if the doc mentions it;

Cheers,

[lmorg]

with your permission and based on the discussion above, I will create independent versions of -compose & typescript - modules

Please do :)

[tiymat]

Just wanted to add that I recently created a module and was also a bit taken aback by the lack of an uninstall command.

Also, I couldn't find any documentation about module.json and am curious about its Dependencies section. Does anyone know if its Optional and Required lists are meant to contain murex modules, or murex packages, or something else?

[lmorg]

Pushed an update to add a remove option to murex-package.

[lmorg]

Just wanted to add that I recently created a module and was also a bit taken aback by the lack of an uninstall command.

Also, I couldn't find any documentation about module.json and am curious about its Dependencies section. Does anyone know if its Optional and Required lists are meant to contain murex modules, or murex packages, or something else?

Required is an array of strings. Each item is a command that your module requires. So it's checked against loaded aliases, functions, builtins, and external executables (in $PATH).

Optional does nothing at this stage. I've added it so that it can be used, albeit I haven't yet written any code to surface such notices nor even decided the best way to display such notices.

[tiymat]

Thanks for the clarification.

[lmorg]

I've now added a new feature to murex-package called "new". eg

What this does is creates a package directory (in this case foo) containing:

» murex-package cd foo
» tree
.
├── bar.mx
├── module.json
└── package.json

It initialises git for that directory (though obviously cannot set an upstream origin) and updates ~/.murex_modules/packages.json to include the new package too.

[tiymat]

Looks very convenient, now I just need an idea for another package to try it out!

[lmorg]

Additionally added a git parameter to murex-package. eg

[orefalo]

Other murex packages -

You have plenty here -

https://github.com/search?q=murex-module-&type=repositories

[orefalo]

Interesting... the latest develop breaks with a panic

➜  ./murex
Loading profile `.murex_preload`
Loading module `jump/jump`
Loading module `murex-module-dockercompose/docker-compose`
Loading module `murex-module-free/free`
Loading module `murex-module-grc/grc`
Loading module `murex-module-javahome/javahome`
Loading module `murex-module-orefalo/profile`
Loading module `murex-module-orefalo/sublime-link`
Loading module `murex-module-orefalo/k9s`
Loading module `murex-module-orefalo/starship-config`
Loading module `murex-module-orefalo/local-docker`
Loading module `murex-module-starship/starship`
Loading module `murex-module-tildekey/tildekey`
Loading profile `.murex_profile`
panic: fork.FileRef == nil in (fork *Fork).Execute()

goroutine 1 [running]:
github.com/lmorg/murex/lang.(*Fork).Execute(0x0?, {0x1400038a750?, 0x23?, 0x1050b0ff1?})
	/Users/orefalo/GitRepositories/murex-related/murex/lang/fork.go:225 +0x7f4
github.com/lmorg/murex/shell.writeTitlebar()
	/Users/orefalo/GitRepositories/murex-related/murex/shell/prompt.go:77 +0x150
github.com/lmorg/murex/shell.ShowPrompt()
	/Users/orefalo/GitRepositories/murex-related/murex/shell/shell.go:165 +0x34c
github.com/lmorg/murex/shell.Start()
	/Users/orefalo/GitRepositories/murex-related/murex/shell/shell.go:100 +0x2d4
main.startMurex()
	/Users/orefalo/GitRepositories/murex-related/murex/main.go:153 +0x5c
main.main()
	/Users/orefalo/GitRepositories/murex-related/murex/main.go:40 +0x5c

[lmorg]

Thanks for catching. I'll push a fix in an hour.

[lmorg]

@orefalo fix deployed to develop

[orefalo]

Just referencing this article - https://antonz.org/writing-package-manager/
it shows how to build a module dependency system in golang.
No rush - low priority

maybe this ticket should be turned into a discussion

[lmorg]

Resolving dependencies is the easy part. What we need to consider is what dependencies we want to resolve.

personally I feel resolving dependencies on other modules brings limited value to a shell, because the power of a shell lives and dies by the commands available to it.

we can resolve dependencies to aliases, functions and even builtins, but what about external executables?

this is where the real problem lies. External executables can be installed in a thousand different ways. Whether it’s operating system differences, differences in distros, even differences on the same platform (eg Ubuntu could have executables installed via snap, Docker, apt, make install, go install, npm, pip, or even just good old wget).

And to complicate things, the package name might not even be the same. Eg on Ubuntu in apt it might be named “foo-lang” but in Centos it might be in yum as “foo”.

Given Murex is cross platform, that makes a massive number of options to calculate. Managing that would be a full time job in itself.

So we need either a really elegant way of managing that, but which is reliable - though I don’t think such a way exists. Or we need to descope external executables.

If external executables are descoped then that doesn’t leave a whole lot left to manage. And thus the usefulness of having dependencies managed become pretty weak. So I’d want to understand the business case (to borrow a corporate term) behind dependency resolution. Ie what’s the problem we are trying to solve? Builtins are already handled via the Murex version constraints field which was shipped in v5.0. Aliases are pretty limited in Murex. So that only really leaves functions.

Thus this leaves us with with one final question: people really writing functions in one module that depends on another module that isn’t shipped in the same package? The idea of packages was to group modules of the same spiritual domain and/or have inter-dependent functions. So does that design go far enough to save the need for dependency resolution?

[orefalo]

I now better understand your position.
IMHO - resolving executable is beyond the scope

If external executables are descoped then that doesn’t leave a whole lot left to manage. And thus the usefulness of having dependencies managed become pretty weak

To be honest, while responding to your message, I was under the impression that module dependencies were necessary.
However, after conducting research within the fisher - a package manager for fish ecosystem, I must admit that you are correct.

With a few exceptions, modules are usually self-contained and intended for user interface. In other words, I have discovered few utility modules that murex doesn't already cover functionally.

one of which is:
https://github.com/jorgebucaran/getopts.fish - I've been trying to port this one lately, no success

other reusable library I can think of: ncurses like prompt and table mgmt

I am afraid it's not sufficient to make a case...

So yeah, topic closed - at least from my position.