Unable to create .json configuration files following the documentation of Gradle plugin

[jvmusin]

Describe the bug
When I ./gradlew -Pagent run an application, do some operations with it and close it not gracefully (imagine an application like while (true) { ... } and killing it), the agent creates .json files not in build/native/agent-output/run, but in build/native/agent-output/session-4671-20220628T174143Z (of smth like that). Because of that, the following task ./gradlew metadataCopy --task run --dir src/main/resources/META-INF/native-image fails saying java.io.FileNotFoundException: /path-to-the-project/build/native/agent-output/run/reflect-config.json (No such file or directory).

To Reproduce

1. Take any project with a plugin id("org.graalvm.buildtools.native") version "0.9.12" and graalvm-ce-java11-22.1.0.
2. Use main function like fun main() { while(true) {} }
3. Run ./gradlew -Pagent run to produce configs
4. Run ./gradlew metadataCopy --task run --dir src/main/resources/META-INF/native-image to copy configs to resources (you'll see an exception here)

Expected behavior
Config files are created and copied to resources folder.

Logs

Rustam.Musin@UNIT-2257 native-image-test % ./gradlew -Pagent run
[native-image-plugin] Instrumenting task with the native-image-agent: run
[native-image-plugin] Instrumenting task with the native-image-agent: test

> Task :compileKotlin
w: /Users/Rustam.Musin/my/native-image-test/src/main/kotlin/Main.kt: (1, 10): Parameter 'args' is never used
w: /Users/Rustam.Musin/my/native-image-test/src/main/kotlin/Main.kt: (3, 5): Unreachable code
w: /Users/Rustam.Musin/my/native-image-test/src/main/kotlin/Main.kt: (7, 5): Unreachable code
<==========---> 80% EXECUTING [3s]
> :run
^C%                                                                                                                                                                                                                     Rustam.Musin@UNIT-2257 native-image-test % ./gradlew -Pagent run                                                           
[native-image-plugin] Instrumenting task with the native-image-agent: run
[native-image-plugin] Instrumenting task with the native-image-agent: test
<==========---> 80% EXECUTING [5s]
> :run
^C%                                                                                                                                                                                                                     Rustam.Musin@UNIT-2257 native-image-test % ./gradlew metadataCopy --task run --dir src/main/resources/META-INF/native-image

> Task :metadataCopy FAILED
[native-image-plugin] Toolchain detection is disabled, will use GraalVM from /Users/Rustam.Musin/Library/Java/JavaVirtualMachines/graalvm-ce-java11-22.1.0/Contents/Home.
java.io.FileNotFoundException: /Users/Rustam.Musin/my/native-image-test/build/native/agent-output/run/reflect-config.json (No such file or directory)
        at java.base/java.io.FileInputStream.open0(Native Method)
        at java.base/java.io.FileInputStream.open(FileInputStream.java:219)
        at java.base/java.io.FileInputStream.<init>(FileInputStream.java:157)
        at java.base/java.io.FileInputStream.<init>(FileInputStream.java:112)
        at java.base/sun.net.www.protocol.file.FileURLConnection.connect(FileURLConnection.java:86)
        at java.base/sun.net.www.protocol.file.FileURLConnection.getInputStream(FileURLConnection.java:184)
        at java.base/java.net.URL.openStream(URL.java:1165)
        at org.graalvm.nativeimage.builder/com.oracle.svm.core.configure.ConfigurationParser.openStream(ConfigurationParser.java:55)
        at org.graalvm.nativeimage.builder/com.oracle.svm.core.configure.ConfigurationParser.openReader(ConfigurationParser.java:76)
        at org.graalvm.nativeimage.builder/com.oracle.svm.core.configure.ConfigurationParser.parseAndRegister(ConfigurationParser.java:70)
        at org.graalvm.nativeimage.configure/com.oracle.svm.configure.config.ConfigurationFileCollection.loadConfig(ConfigurationFileCollection.java:164)
        at org.graalvm.nativeimage.configure/com.oracle.svm.configure.config.ConfigurationFileCollection.loadTypeConfig(ConfigurationFileCollection.java:157)
        at org.graalvm.nativeimage.configure/com.oracle.svm.configure.config.ConfigurationFileCollection.loadReflectConfig(ConfigurationFileCollection.java:120)
        at org.graalvm.nativeimage.configure/com.oracle.svm.configure.config.ConfigurationFileCollection.loadConfigurationSet(ConfigurationFileCollection.java:150)
        at org.graalvm.nativeimage.configure/com.oracle.svm.configure.ConfigurationTool.generate(ConfigurationTool.java:288)
        at org.graalvm.nativeimage.configure/com.oracle.svm.configure.ConfigurationTool.main(ConfigurationTool.java:100)

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':metadataCopy'.
> Process 'command '/Users/Rustam.Musin/Library/Java/JavaVirtualMachines/graalvm-ce-java11-22.1.0/Contents/Home/bin/native-image-configure'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 1s
1 actionable task: 1 executed
Rustam.Musin@UNIT-2257 native-image-test % 

System Info (please complete the following information):

• OS: macOS Monterey
• GraalVM Version: 22.1 CE
• Java Version: 11
• Plugin version: native-gradle-plugin:0.9.12

[jvmusin]

native-image-test.zip
Here is a sample project in case you need it.

[jvmusin]

I used a documentation here: https://graalvm.github.io/native-build-tools/latest/gradle-plugin.html#agent-support

[jvmusin]

Btw after reverse engineering a bit, I found a WA: you can replace --task run with --task run/session-4671-20220628T174143Z

[lazar-mitrovic]

That session folder is used in order to make sure that concurrent agent runs don't overwrite each-others outputs -- it should at a later stage be merged using a dedicated task, however SIGKILL prevents that task from ever executing.

What it looks to me is that we need to implement a watchdog that would finalize agent processing.
WDYT @melix?

[melix]

I'm not sure that would be easy to do. Hitting CTRL+C will interrupt the Gradle build and it is expected, from my POV, that it doesn't do more. Maybe we should document this behavior instead, and recommend to have a stop endpoint in the application to shutdown it cleanly (e.g [https://docs.micronaut.io/latest/guide/#stopEndpoint](what Micronaut offers).

[jvmusin]

This also happens when an application ends with an exit code != 0 or with an exception.

[melix]

If the app ends with an different exit code, you can still use --continue to get past.

[jvmusin]

Even when I use ./gradlew -Pagent run --continue, it creates config files in run/session-... subdirectory and not in just run.

[jvmusin]

Can you please mark NativeImageOptions.getAgent() and DeprecatedAgentOptions class and its methods as deprecated?

[XhstormR]

I had the same problem.

root@e3022b4218a7:/app# gradle metadataCopy --task run --dir src/main/resources/META-INF/native-image

> Configure project :
Kotlin DSL property assignment is an incubating feature.

> Task :metadataCopy FAILED
java.io.FileNotFoundException: /app/build/native/agent-output/run/reflect-config.json (No such file or directory)
        at java.base/java.io.FileInputStream.open0(Native Method)
        at java.base/java.io.FileInputStream.open(FileInputStream.java:216)
        at java.base/java.io.FileInputStream.<init>(FileInputStream.java:157)
        at java.base/java.io.FileInputStream.<init>(FileInputStream.java:111)
        at java.base/sun.net.www.protocol.file.FileURLConnection.connect(FileURLConnection.java:86)
        at java.base/sun.net.www.protocol.file.FileURLConnection.getInputStream(FileURLConnection.java:189)
        at java.base/java.net.URL.openStream(URL.java:1161)
        at org.graalvm.nativeimage.builder/com.oracle.svm.core.configure.ConfigurationParser.openStream(ConfigurationParser.java:56)
        at org.graalvm.nativeimage.builder/com.oracle.svm.core.configure.ConfigurationParser.openReader(ConfigurationParser.java:77)
        at org.graalvm.nativeimage.builder/com.oracle.svm.core.configure.ConfigurationParser.parseAndRegister(ConfigurationParser.java:71)
        at org.graalvm.nativeimage.configure/com.oracle.svm.configure.config.ConfigurationFileCollection.loadConfig(ConfigurationFileCollection.java:164)
        at org.graalvm.nativeimage.configure/com.oracle.svm.configure.config.ConfigurationFileCollection.loadTypeConfig(ConfigurationFileCollection.java:157)
        at org.graalvm.nativeimage.configure/com.oracle.svm.configure.config.ConfigurationFileCollection.loadReflectConfig(ConfigurationFileCollection.java:120)
        at org.graalvm.nativeimage.configure/com.oracle.svm.configure.config.ConfigurationFileCollection.loadConfigurationSet(ConfigurationFileCollection.java:150)
        at org.graalvm.nativeimage.configure/com.oracle.svm.configure.command.ConfigurationGenerateCommand.generate(ConfigurationGenerateCommand.java:273)
        at org.graalvm.nativeimage.configure/com.oracle.svm.configure.command.ConfigurationGenerateCommand.apply(ConfigurationGenerateCommand.java:60)
        at org.graalvm.nativeimage.configure/com.oracle.svm.configure.ConfigurationTool.main(ConfigurationTool.java:84)

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':metadataCopy'.
> Process 'command '/opt/java/graalvm/bin/native-image-configure'' finished with non-zero exit value 1

BUILD FAILED in 1s
1 actionable task: 1 executed

root@e3022b4218a7:/app# ll /app/build/native/agent-output/run/
total 0
drwxr-xr-x 3 root root  96 Aug 27 12:53 ./
drwxr-xr-x 3 root root  96 Aug 27 12:53 ../
drwxr-xr-x 9 root root 288 Aug 27 12:53 session-6549-20230827T125328Z/
root@e3022b4218a7:/app# ll /app/build/native/agent-output/run/session-6549-20230827T125328Z/
total 28
drwxr-xr-x 9 root root  288 Aug 27 12:53 ./
drwxr-xr-x 3 root root   96 Aug 27 12:53 ../
drwxr-xr-x 2 root root   64 Aug 27 12:53 agent-extracted-predefined-classes/
-rw-r--r-- 1 root root  113 Aug 27 12:53 jni-config.json
-rw-r--r-- 1 root root   65 Aug 27 12:53 predefined-classes-config.json
-rw-r--r-- 1 root root    4 Aug 27 12:53 proxy-config.json
-rw-r--r-- 1 root root 6865 Aug 27 12:53 reflect-config.json
-rw-r--r-- 1 root root 1013 Aug 27 12:53 resource-config.json
-rw-r--r-- 1 root root   71 Aug 27 12:53 serialization-config.json
root@e3022b4218a7:/app#

Replace run with run/session-6549-20230827T125328Z to successfully execute the task:

root@e3022b4218a7:/app# gradle metadataCopy --task run/session-6549-20230827T125328Z --dir src/main/resources/META-INF/native-image

> Configure project :
Kotlin DSL property assignment is an incubating feature.

Deprecated Gradle features were used in this build, making it incompatible with Gradle 9.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

For more on this, please refer to https://docs.gradle.org/8.3/userguide/command_line_interface.html#sec:command_line_warnings in the Gradle documentation.

BUILD SUCCESSFUL in 1s
1 actionable task: 1 executed
root@e3022b4218a7:/app#