-
-
Notifications
You must be signed in to change notification settings - Fork 1.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Feature: add :doc
command to nix repl
#3904
Comments
:doc
command to nix-repl:doc
command to nix repl
Maybe let's start with |
@Ericson2314 if that is the preference, we should probably be making the repl extensible so we can push back the syntax/etc discussion about a documentation format while a prototype is made available. Currently it is possible to get the documentation of a function definition just by linearly traversing the AST backwards looking for comments and stopping once you hit non-trivia tokens, although this is not yet implemented in Nix to the best of my knowledge. This trivial solution might accidentally hit some comments that are not documentation, but that is perhaps a worthwhile sacrifice, since a real documentation comment syntax could be built on this feature in nixpkgs anyway, assuming it collects all the comments for a given function: a regex could be applied to catch all the ones in the "doc" format. I believe @infinisil sketched out some ideas for how the API for such a thing could work. I have most of the bits to make a (not nearly as ergonomic/nice) prototype without any changes in Nix itself by shelling out to my own tool, but I don't have the C++ background to know how to begin on doing it natively. This would integrate into a custom commands feature well. |
The idea I had was that Nix could be extended to have new builtins for adding annotations to expressions like: builtins.withAnnotation <key> <value> <expression>
# ^^ Returns a value that behaves the same as <expression> but with the <key> annotation set to <value>
builtins.getAnnotation <key> <expression>
# ^^ Gets the <key> annotation of an expression. Returns null if none was set This mechanism would allow adding annotations to expressions without changing their behavior, but the annotations can still be accessed later. Nix could then use this to automatically transform comments into annotations. E.g. the following code: {
# This does foo
foo = "foo";
} would be transformed to {
foo = builtins.withAnnotation "comment" "This does foo" "foo";
} Or maybe even {
foo = builtins.withAnnotation "attributeName" "foo" (builtins.withAnnotation "comment" "This does foo" "foo");
} And tools like |
I assume that trick is fine for purity since unless you are putting annotations into your derivations or something, changing an annotation wouldn't result in different output? Looks good in any case! |
I am poking around on implementing this without extending Nix and there is a flaw that makes this effort sort of a futile exercise: it doesn't appear that positions of functions are exposed to Nix expressions. It's still possible to use unsafeGetAttrPos, but that requires that functions are attributes in a set, and it doesn't find where the function actually is defined if it is e.g. imported from another file, instead only finding where the attribute position is defined. This feature appears to require support in Nix itself. |
This is useful for a potential pure-Nix implementation of NixOS#3904.
Update: I may or may not have made irresponsible choices tonight but I did write a prototype nix plugin that adds the documentation functionality as an (evil and bad) builtin using the existing nix-doc parsing infrastructure (unfortunately plugins cannot extend the REPL). In doing so, I realized something that is going to be problematic for any possible solution: Nix makes it really easy to compose functions everywhere, which means that it is often very hard to find the actual source of the function because so many functions are wrapped. I was not expecting to have issues with this when I have access to the actual objects by running in-process in Nix, but here I am: take the example from above of fetchFromGitHub:
Edit: If you want to try out this prototype that shows the limitations of a trivial solution, it's available at https://github.com/lf-/nix-doc. |
This pr could be relevant #1652. It was a bit of a hack though. |
@roberth gee, the desire to have this feature really is everpresent haha. I really don't want it to die by bikeshedding this time around, as it is incredibly useful to have as a feature, even if a simple implementation returns some garbage sometimes. I will say, my implementation cannot die to the bikeshed because it cannot in good conscience ever be merged ;-) |
Update: Well, I made it happen: lf-/nix-doc@21df582 plus #3934 constitute essentially a full implementation of the proposal above. This bug is, in my view, still not fixed because it realistically probably should be in Nix itself, and the documentation markup language bikeshed has not been painted yet to the best of my knowledge. Also, we need a good way of dealing with wrapper functions, such as
|
We now have a
This could be extended by docstring annotations on functions, however we'll have to think about whether this is the right way to improve discoverability in Nix. Adding something like the NixOS module system as a language feature would get us documentation for free (see https://gist.github.com/edolstra/29ce9d8ea399b703a7023073b0dbc00d). |
@edolstra I find that although there is existing html documentation for the library and built-in functions, I can't use it very effectively since it's on a massive page so ctrl-f gets a bunch of extraneous results, the page takes a couple of seconds to jump to the section on load, etc. I can keep my focus much better by doing it within Nix (from experience with my plugin), so even if this is resolved, it would be disappointing to have html documentation as the only option. Further, often the web based nixpkgs library documentation is not enough because I'm working in the Haskell infrastructure which largely isn't documented except in the source so I just have to read the source code regardless. Another thing that's been extremely valuable about my plugin is that it dumps parameter names, which is the thing I really struggle with most of the time: "what do I have to give this thing". This is, in my view, more useful than the annotation. What I would also enjoy is the ability to add custom commands, like one that directly brings up the given function in my editor, figuring out where it's actually defined and going directly to the line. #3934 is bringing out the big guns for this, allowing pretty much anything to be written, but it would also be good to be able to somehow write commands in something more accessible to users than the partially documented C++ API. Nix will have trouble with go to definition since we have a lot of generic function wrappers in nixpkgs that end up hiding where the actual implementation I'm looking for is defined (so it's better to get the definition site of the attribute, where the wrapper is called, rather than the lambda). I'd like it if I could traverse these with a documentation tool somehow, maybe through tracing through them? The specific big use case I'm eyeing is being able to jump directly to the Nix file for a given package in nixpkgs, though there's also some wrapped library functions that benefit from this. |
Implemented in Lix. Feel free to cherry-pick it. ^.^ https://gerrit.lix.systems/c/lix/+/564 |
I don't think it's a good solution to have "doc-comments" in I generally don't think it is a good idea to maintain two parsers in official implementation. Why you like to parse the file again, by another parser, and heuristically find the result? Instead of some updates for existing parser? |
We agreed on that with the team when we discussed it last week. I opened this issue to discuss the option of adopting Lix's solution, but we have decided that it's better not to, for the reasons you describe. |
Is your feature request related to a problem? Please describe.
I am always having to look up function documentation in the nixpkgs repository with either
ripgrep
or the tool I wrote because of this problem,nix-doc
, but neither is perfect:ripgrep
finds a lot of overridden copies of the function, and you still often have to jump between several files to decipher what's required to call a function.By comparison,
nix-doc
uses a heuristic of looking in the AST for function definitions with comments above them to eliminate overrides, but this heuristic misses functions that are defined in their own file such asfetchFromGitHub
, which is a problem I don't think I can practically solve in a tool that doesn't intend to understand the Nix language.The Nix repl currently has the information to provide a better documentation experience as it knows where the actual function is defined through
builtins.unsafeGetAttrPos
, but it is not immediately obvious how it can nicely be extended to call some external tool to parse the nix file and find the comments above the function and output them.Describe the solution you'd like
I want to have the following session in
nix repl
(also imagine it has colour):Describe alternatives you've considered
It is /maybe/ possible to do this in nixpkgs, but you'd need a nix parser in nixpkgs OR make nix expose the comments in some attribute (discussed on IRC; plausible). Ergonomically it is probably superior to have this built into
nix repl
or to makenix repl
support some kind of profile mechanism or other trick to add more:command
s.Additional context
cc @infinisil we discussed this on IRC in #nix-lang a couple of weeks ago
The text was updated successfully, but these errors were encountered: