envrc runs direnv synchronously

[sellout]

Some .envrc files are slow to execute. E.g., anything with use flake or use nix could take quite a while before completing. This currently locks all of Emacs. I should still be able to work on other projects and even track the *envrc* buffer on the current project. I think this is more important than preventing any use of the environment before it has switched over.

A more advanced handler (beyond simply going async) could perhaps explicitly block on the async process until it either 1. completes (and then silently unblocks) or 2. observes “direnv: (…) is taking a while to execute. Use CTRL-C to give up.” in the output (and then brings *envrc* to the front and processing continues async).

But I’d be more than happy with just a fire-and-forget.

[psionic-k]

While blocking all of Emacs would be annoying, I would not avoid blocking shell commands. This dependency should probably be explicit. Probably Emacs needs an analog to buffer locals and the integration at the project level.

[purcell]

Tricky. I totally get it, and have the same issue myself at times, but this was a deliberate choice due to the difficulty of doing the right thing if the env is loaded asynchronously: loading the right environment is something the user wants because of a subsequent step that requires it, which could either be an interactive action or a programmed one.

In the latter case, I'm particularly thinking about when the env is initialised during mode initialisation, where it might be relied upon by other code added to the mode hooks. This is quite common, I think: it may or may not be where you're encountering blocking. (I know that it's the most common case for me, as a Nix user.)

I don't know if loading asynchronously and then triggering a custom hook would be feasible as a pattern for accomplishing the same things in practice.

It's easier to imagine reloading envs asynchronously, or in a more interactive way, because it's usually triggered interactively.

Ultimately, slow envrc files are the minority, and can often be addressed directly via other means — in the case of Nix, retaining GC roots can make full reloads less likely. I'm a bit wary of making the code complicated, unpredictable or hard to debug. For starters, what if multiple buffers start trying to asynchronously load the same env — now you have to handle locking etc.

Any thoughts on these trade-offs?

[sellout]

For the simple case, what about (defcustom envrc-load-async nil …? With a docstring that recommends (similarly to the many of Projectile’s defcustoms) setting it in a .dir-locals.el adjacent to the relevant .envrc rather than via Customize.

But you’re right – as soon as you do any of that, you need locking …

I’ll have to pay attention to when this gets in my way. But I tend to use a single Emacs instance for development across multiple machines, so when something is blocking, it’s suddenly painful. I think I mostly trigger it by doing something like nix flake update in async-shell-command, then going off to do something in another project and finding my Emacs blocked at some point along the way. (Maybe when refreshing Magit after the update?)

[psionic-k]

Per-buffer blocking would be totally acceptable and correct. How could envrc cleanly take control of the buffer loading sequence and restore it without blocking?

Most packages expect to read the environment once, during the buffer hooks. Initializing the environment late would require hooks to be idempotent or for dependents to re-read the environment. I think we will all collectively spend more time on the bugs that emerge than waiting for individual buffers.

As for what might go wrong by suspending buffer loads until the asynchronous environment initialization finishes, the most challenging case would be multiple buffers opening at once in workspace-like flows. IIRC nix locking and really any competent envrc process will already handle multiple shells arriving in the same direnv. On the elisp side, we have buffer locals to isolate the side effects of loads completing, and competent packages shouldn't pollute other buffers.

In short, I think per-buffer blocking is the right choice, but an implementation question.

[psionic-k]

If the blocking behavior has caching (like Nix direnv integration), vterm can operate as a workaround to blocking.

[Mic92]

Duplicate of #6

[purcell]

Yep, closing as a duplicate, sorry I didn't spot that sooner -> let's continue discussion in the older issue (#6).