Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Plugin support #1739

Merged
merged 63 commits into from
Feb 11, 2022
Merged

feat: Plugin support #1739

merged 63 commits into from
Feb 11, 2022

Conversation

MikaelMayer
Copy link
Member

@MikaelMayer MikaelMayer commented Jan 20, 2022

This PR creates an extensible plugin structure, which is primarily used to provide Dafny with extra Rewriters .
It will discharge Dafny core developers by letting any Dafny developer to experiment with their own pipelines and rewritings without the need to modify Dafny core.

Namely:

Dafny pipeline

  • **I created a new PluginRewriter that extends IRewriter
  • **I created the class to Plugin\Rewriter that is wrapped by a PluginRewriter. Plugin\Rewriter is a class that plugins are going to extend.
  • I created the class Microsoft.Dafny.Plugin that takes care of loading assemblies and supply them with command-line arguments.
  • I created the class Microsoft.Dafny.Plugins.Configuration that plugins should extend, so that they can optionally handle command-line arguments, and especially override methods such as GetRewriters.
  • I added in DafnyOptions the option plugin, which consists of the path to a dll + optionally, after a comma, a list of space-separated arguments. I modified the help accordingly. This option can be repeatedly provided.
  • This option automatically loads all the provided assemblies as a list of Plugin DafnyOptions.O.Plugins, or fail immediately if plugins cannot be loaded, or if their initialization fail. It will instantiate a single instance of type Microsoft.Dafny.Plugins.Configuration which will be responsible for handling command-line arguments and providing the extra Rewriters. If no Configuration is provided, Dafny will just try to instantiate all the Rewriters it will find. Kind of a lightweight plugin mechanism.
  • In the Resolver, when it gather all internal Rewrite classes, it now goes through all the plugins from DafnyOptions.O.Plugins and recover every plugin-defined Rewriter wrapped in PluginRewriter by calling GetRewriters on the plugin, and add them to the list rewriters It does not check for correct version though, I'm not sure how we should approach that, except to add a mention in the README.md
  • I added two tests in PluginsTest.cs in DafnyPipeline.Test that verifies it can load a dummy Rewriter (credits to @keyboardDrummer for the code to do so with the CSharpCompiler which I heavily copied) and it is correctly called during resolution. It also tests plugins without Configuration (automatically loaded)

Dafny language server

  • To ensure any error in loading such assembly is visible in the VSCode extension, I created a routine that fetches the first non-trivial token, which is used to report the error, rather than Token.NoToken
  • I added an option to the DafnyLanguageServer, namely --dafny:plugin:0= which serves the same purpose as the /plugins: option of Dafny above. This option can be launched in VSCode by adding "dafny.languageServerLaunchArgs": ["--dafny:plugins=absolute_path_to_plugin.dll"] to the settings.json
  • Rather than modifying the settings.json that way, I also updated the VSCode extension so that one can provide this argument in the settings rather the settings.sjon: Here is the VSCode PR
  • Updated the README appropriately
  • I added two tests, "Various\PluginsTest.cs" and "Various\PluginsAdvancedTest.cs" that tests that the DafnyLanguageServer loads an assembly correctly. It also tests that it is possible to put a comma in the arguments themselves. The second is a realistic use case in which it should report extern methods that are not tested.

Note: We will be able to build on top of this PR to use the same option to recover compilers, by adding a method GetCompiler or GetCompilers to the class Configuration.

Things to know when reviewing this PR

  • PostResolve was renamed PostResolveIntermediate because many things happen after the original PostResolve in the resolver. A new PostResolve hook now really happens at the end of the resolver
  • Comments in Plugin\Rewriter.cs and Plugin\Configuration.cs clearly describe what API we would provide to customers (@robin-aws it should be crystal clear)
  • There are two tests called PluginsTest, one in DafnyPipeline.Test and two other in DafnyLanguageServer.Test. I did not find where I could merge the library loading mechanism because they don't have a test project in common, so these methods are duplicated. I think this is fine, we might want to do other experiments with external resolvers tied to DanfyLanguageServer.
  • I did not find a way to get rid of the temporarily allocated DLL since it's loaded. I do remove it from DafnyOptions in DafnyLanguageServer to avoid conflicts between tests.
  • I decided to throw an exception when instantiating options, if the plugin fails to load, rather than later, because we might want to use these plugins in other places (maybe the C# preprocessor at some point for pre-parsing?)

PR that fixes the plugin argument parsing so that it's simple: #1817

@cpitclaudel
Copy link
Member

Can you say how this PR interacts with #1708 ? Does is replace it?

Copy link
Member

@camrein camrein left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good so far. Is there a reason you didn't create a test for the language server?

Source/Dafny/DafnyOptions.cs Outdated Show resolved Hide resolved
Source/DafnyLanguageServer/Language/CompilerOptions.cs Outdated Show resolved Hide resolved
Source/DafnyLanguageServer/README.md Outdated Show resolved Hide resolved
}
}
}, new JsonSerializerOptions() { WriteIndented = true });
await File.WriteAllTextAsync(temp + ".runtimeconfig.json", configuration + Environment.NewLine);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the *.runtimeconfig.json file necessary for dynamically loaded assemblies?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know, I just copied-pasted the code of @keyboardDrummer . I removed it and it works, thanks for reporting.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perfect. I guess you've copied this from @keyboardDrummer's code that test's the shutdown of the language server if the parent process quits. The difference to his implementation, is that you're creating a DLL that's loaded by a running process while his implementation is used to spawn a new process. However, I'm not sure if the *.runtimeconfig.json file is required if the application is launched with the dotnet command; it sure is when calling the executable directly.

Copy link
Member

@keyboardDrummer keyboardDrummer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this PR also support letting the language server show errors generated by existing backends, such as the error here? https://github.com/dafny-lang/dafny/blob/master/Source/Dafny/Compilers/Compiler-Csharp.cs#L2469

I think we should build one solution that enables error reporting both for existing and custom back-ends. What are your thoughts?

Copy link
Member

@robin-aws robin-aws left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just flagging that I'm especially interested in making sure we're crystal clear on the API here - consider this a ad-hoc mechanism for saying "I'm making myself a required reviewer" :)

@MikaelMayer
Copy link
Member Author

Looks good so far. Is there a reason you didn't create a test for the language server?

No good reason and plenty of good reasons to actually do this. I'm going to work on that. One of the big catch of a plugin is that it can theoretically write to standard output, which messes up the disagnostics for the LSP. For now, I'm thinking of putting a comment on the class that plugins need to extend, because error reporters directly print to the console and we don't want to change that I guess.

I think we should build one solution that enables error reporting both for existing and custom back-ends. What are your thoughts?

I agree. Since the compilers are going to be refactored in #1708, it's better to wait to delegate that mechanism there, and @cpitclaudel and I thought about using generic "plugins" arguments, from which both the Resolver phase and the Compiler phase are going to pull their extra phases; thus, it will be easy to define for every compiler an IRewriter (used in Resolver) that will take care of doing exactly what you want, and nothing will need to be changed on this PR.

Just flagging that I'm especially interested in making sure we're crystal clear on the API here - consider this a ad-hoc mechanism for saying "I'm making myself a required reviewer" :)

Copy that, required reviewer ! Here are the pros and the cons of the current approach:

  • One single DLL for rewriters and compilers in the future.
  • One simple class to extend, an error reporter provided, and later the post-resolved tree in a method to override, and where to report errors.
  • Plugins can manipulate the actual Dafny tree directly.
  • Works for both DafnyDriver and DafnyLanguageServer
  • It will depend on the version of Dafny itself, especially if we change the AST implementation (having a tool that takes care of parsing itself would have less of this problem). I'll focus on this next so that the plugin and Dafny are aware of version compatibility.
  • We offer an API for pre-resolve, which might not be desirable. (we could split the IRewriter)
  • We are not offering a way to "prioritize" IRewriters, but this could be added later.
  • We rely on the IRewriters to not modify the AST too much so that it would break further verification and compilation. However, in most places, we ensure that expressions are resolved only once, and typed only once.

@keyboardDrummer
Copy link
Member

keyboardDrummer commented Jan 25, 2022

@cpitclaudel and I thought about using generic "plugins" arguments, from which both the Resolver phase and the Compiler phase are going to pull their extra phases

I'm not convinced there should be extra resolver phases. Why don't we only use the ICompiler/ICodeGenerator interface for plugins? I feel less plugin surface is better.

/// If the plugin defines no Configuration, then Dafny will instantiate every sub-class
/// of Rewriter from the plugin, providing them with an ErrorReporter as the first and only argument.
/// </summary>
public abstract class Rewriter {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

New API looks good :)

@@ -2007,6 +2080,22 @@ class Induction_Visitor : BottomUpVisitor {
}
}
}

class PluginRewriter : IRewriter {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How come this class and Plugins.Rewriter had to be separate classes again?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A Dafny rewriter (IRewriter) can support more phases than a plugin rewriter (which only supports PostResolve)
Therefore, we need to encapsulate any plugin-defined Rewriter into the (more internal) Dafny IRewriter.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would have thought a single abstract class with some virtual methods marked as internal and some as public (the ones we want to expose) would have been sufficient. How come it's not?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would have been sufficient. However, it may not be a good practice for a plugin system because all the "internal" methods would still be in the Plugin namespace, exposed to anyone looking at the source code - which I expect from plugins writers since they need to have their plugin in sync with the Dafny source code.
I prefer to really split what we offer between two things, one for the plugin, and one for internal. That way, if we want to change how the internal work, we are not stuck by a public API that we can always re-route to the new constructs.

await SetUpPlugin();
}

protected override string LibraryCode =>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure what I prefer, but think we could also put this library code in a separate .cs file that's not included in the build but retrieved as a resource using assembly.GetManifestResourceStream(resourceName). That way you would at least get syntax highlighting and parse errors when editing the file.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Implemented.

protected DiagnosticsReceiver DiagnosticReceiver;
protected string LibraryPath;

private static readonly object PluginSync = new { };
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this used anywhere?

class AutomaticConfiguration : Configuration {
private Plugin(string path, string[] args) {
pluginPath = path;
assembly = Assembly.LoadFrom(pluginPath);
Copy link
Member

@keyboardDrummer keyboardDrummer Feb 8, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nitpick: I would even do this work in the Load method and then pass the assembly to the constructor.

Also, consider using a record to remove the need for a constructor, like so:

public record Plugin(Assembly Assembly, string[] Args) {
  public Configuration PluginConfiguration { get; init; } = LoadConfiguration(Assembly, Args);

  public static Plugin Load(string pluginPath, string[] args) {
    return new Plugin(Assembly.Load(pluginPath), args);
  }

  private static Configuration LoadConfiguration(Assembly assembly, string[] args) {
    var rewriterTypes = CheckPluginForRewriters(assembly);
    Configuration configuration = null;
    foreach (var configurationType in GetConfigurationsTypes(assembly)) {
      if (configuration != null) {
        throw new Exception(
          $"Only one class should extend Microsoft.Dafny.Plugins.Configuration from {assembly.Location}. Please remove {configurationType}.");
      }

      configuration = (Configuration)Activator.CreateInstance(configurationType);

      if (configuration == null) {
        throw new Exception($"Could not instantiate a {configurationType} from {assembly.Location}");
      }

      configuration.ParseArguments(args);
    }

    configuration ??= new AutomaticConfiguration(
      rewriterTypes.Select(rewriterType =>
        (ErrorReporter errorReporter) =>
          Activator.CreateInstance(rewriterType, new object[] { errorReporter }) as Rewriter).ToArray());
    return configuration;
  }

  private static System.Type[] CheckPluginForRewriters(Assembly assembly) {
    System.Type[] rewriterTpes = assembly.GetTypes().Where(t =>
      t.IsAssignableTo(typeof(Rewriter))).ToArray();
    // Checks about the plugin to be well-behaved.
    if (!rewriterTpes.Any()) {
      throw new Exception($"Plugin {assembly.Location} does not contain any Microsoft.Dafny.Plugins.Rewriter");
    }

    return rewriterTpes;
  }

  public static IEnumerable<System.Type> GetConfigurationsTypes(Assembly assembly) {
    return assembly.GetTypes()
      .Where(t => t.IsAssignableTo(typeof(Configuration)));
  }

  class AutomaticConfiguration : Configuration {
    private Func<ErrorReporter, Rewriter>[] rewriters;
    public AutomaticConfiguration(Func<ErrorReporter, Rewriter>[] rewriters) {
      this.rewriters = rewriters;
    }

    public override Rewriter[] GetRewriters(ErrorReporter errorReporter) {
      return rewriters.Select(funcErrorReporterRewriter =>
        funcErrorReporterRewriter(errorReporter)).ToArray();
    }
  }
}

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great alternative. I like to put arguments to the class too. Note that it's Assembly.LoadFrom, not Assembly.Load.

Copy link
Member

@camrein camrein left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A bit more nitpicking from my side.

/// The configuration provides the methods to parse command-line arguments and obtain Rewriters
/// </summary>
public record Plugin(Assembly Assembly, string[] Args) {
public PluginConfiguration PluginConfiguration { get; init; } = LoadConfiguration(Assembly, Args);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: init is only necessary if you want that the default initializer can be replaced when creating a new Plugin instance (using the object initializer syntax). For example, it allows doing this:

var plugin = new Plugin(assembly, args) {
  PluginConfiguration = /* some custom configuration */
};

If that's not needed, you can remove the init modifier.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the tip ! I did not know what this init was doing, so your explanation was very useful.


public static IEnumerable<System.Type> GetConfigurationsTypes(Assembly assembly) {
return assembly.GetTypes()
.Where(t => t.IsAssignableTo(typeof(PluginConfiguration)));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the sake of consistency within the same class (single letter identifiers), I'd use type instead of t as the identifier name.

}

private static System.Type[] CheckPluginForRewriters(Assembly assembly) {
System.Type[] rewriterTpes = assembly.GetTypes().Where(t =>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the sake of consistency within the same class (single letter identifiers), I'd use type instead of t as the identifier name.

}

private static System.Type[] CheckPluginForRewriters(Assembly assembly) {
System.Type[] rewriterTpes = assembly.GetTypes().Where(t =>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it necessary to specify System.Type[] explicitly here?

pluginConfiguration ??= new AutomaticPluginConfiguration(
rewriterTypes.Select<System.Type, Func<ErrorReporter, Rewriter>>((System.Type rewriterType) =>
(ErrorReporter errorReporter) =>
Activator.CreateInstance(rewriterType, new object[] { errorReporter }) as Rewriter).ToArray());
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use (Rewriter)Activator.CreateInstance(rewriterType, new object[] { errorReporter }) instead. Alternatively, the LINQ statement .Cast<Rewriter>() might be handy too.


namespace Microsoft.Dafny.LanguageServer {
public static class DafnyLanguageServer {
public static List<string> LoadErrors = new();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Expose this list as a property of type IReadOnlyList<string>.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Then I have to assign the whole property at once in LoadPlugins, which mean I can't make LoadErrors readonly. Is that ok?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can implement the property so it uses a backing field. Making the external API readonly, but the internal mutable. For example you can do this:

private static List<string> loadErrors = new();
public static IReadOnlyList<string> LoadErrors => loadErrors;

keyboardDrummer
keyboardDrummer previously approved these changes Feb 9, 2022
}

pluginConfiguration ??= new AutomaticPluginConfiguration(
rewriterTypes.Select<System.Type, Func<ErrorReporter, Rewriter>>((System.Type rewriterType) =>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the type arguments to Select can be left out.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried to remove it several times (because the IDE suggests it to me), but every time I do, dotnet compiler says it needs that type.

Copy link
Member

@camrein camrein left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some nitpicking from my side


pluginConfiguration ??= new AutomaticPluginConfiguration {
Rewriters =
rewriterTypes.Select<System.Type, Func<ErrorReporter, Rewriter>>((System.Type rewriterType) =>
Copy link
Member

@camrein camrein Feb 10, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The explicitly typed lambda expressions should be redundant due to the type arguments to Select.
However, the requirement for the explicit type arguments might be a good sign that it's best to extract the projection into a separate method in terms of readability:

    pluginConfiguration ??= new AutomaticPluginConfiguration {
      Rewriters = rewriterTypes.Select(CreateRewriterFactory).ToArray()
    };
    return pluginConfiguration;
  }

  private static Func<ErrorReporter, Rewriter> CreateRewriterFactory(System.Type rewriterType) {
    return errorReporter => (Rewriter)Activator.CreateInstance(rewriterType, new object[] { errorReporter });
  }

I leave it up to you if you want to apply this change.


namespace Microsoft.Dafny.LanguageServer {
public static class DafnyLanguageServer {
public static IReadOnlyList<string> LoadErrors = new List<string>();
Copy link
Member

@camrein camrein Feb 10, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just to make sure that you didn't miss a small detail in my suggestion. The implementation I proposed was an expression-bodied getter-only property:

private static List<string> loadErrors = new();
public static IReadOnlyList<string> LoadErrors => loadErrors; // Mind the arrow here

This will effectively compile into something like this:

private static List<string> loadErrors = new();
public static IReadOnlyList<string> LoadErrors {
  get {
    return loadErrors;
  }
}

The current implementation allows changing the value of LoadErrors from outside the DafnyLanguageServer class which I'd like to avoid.

Alternatively, to not change too much of your current implementation, you could also change the field into a property with a private setter:

public static IReadOnlyList<string> LoadErrors { get; private set; } = new List<string>();


namespace Microsoft.Dafny.LanguageServer {
public static class DafnyLanguageServer {
public static IReadOnlyList<string> LoadErrors = new List<string>();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nitpicking as of now, but I'd like to avoid (mutable) static fields within the language server as much as possible.
However, we may want to consider introducing a global state object for the language server in the future for such cases.

@MikaelMayer MikaelMayer enabled auto-merge (squash) February 10, 2022 18:39
@MikaelMayer
Copy link
Member Author

I just enabled auto-squash-and-merge. @camrein let me know if there is anything else I should change, else you can approve this PR.

Copy link
Member

@camrein camrein left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fantastic work!

@MikaelMayer MikaelMayer merged commit c3292b7 into master Feb 11, 2022
@MikaelMayer MikaelMayer deleted the resolving-plugin-support branch February 11, 2022 09:59
@@ -71,7 +71,7 @@ public class ShutdownTest {

// Wait for the language server to kill itself by waiting until it closes the output stream.
await process.StandardOutput.ReadToEndAsync();
Thread.Sleep(300); // Give the process some time to die
await Task.Delay(400); // Give the process some time to die
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not WaitForExitAsync?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know what this does but sometimes a test fail because the process did not die.
@keyboardDrummer ?

}

protected void CleanupPlugin() {
DafnyOptions.O.Plugins.RemoveAt(0);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What does this part do?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When testing a plugin, it adds it to the DafnyOptions.
I remove it from the DafnyOptions after the a test class is finished.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants