investigate graal compiler

[wmorgan]

Things that would be good to know about Linkerd and the new Graal VM:

• Does the Graal JIT compiler improve Linkerd performance and/or memory usage?
• Can Linkerd use the Graal AOT compiler? If so, does itmprove Linkerd performance and/or memory usage?

There's a walkthrough in the first two steps here: https://medium.com/graalvm/graalvm-ten-things-12d9111f307d

[zackangelo]

Did a quick GraalVM compile run to see if there were any obvious issues. Looks like we're doing some reflection-ey/classloader-ey things that it doesn't like:

./native-image --no-server -cp linkerd-1.4.0-SNAPSHOT.jar io.buoyant.linkerd.Main
   classlist:  12,929.97 ms
       (cap):   2,482.33 ms
       setup:   3,897.65 ms
May 15, 2018 3:31:42 PM com.twitter.finagle.http.HttpMuxer$ $anonfun$new$1
INFO: HttpMuxer[/admin/metrics.json] = com.twitter.finagle.stats.MetricsExporter(<function1>)
May 15, 2018 3:31:42 PM com.twitter.finagle.http.HttpMuxer$ $anonfun$new$1
INFO: HttpMuxer[/admin/per_host_metrics.json] = com.twitter.finagle.stats.HostMetricsExporter(<function1>)
RecomputeFieldValue.FieldOffset automatic substitution failed. The automatic substitution registration was attempted because a call to sun.misc.Unsafe.objectFieldOffset(Field) was detected in the static initializer of com.twitter.util.Promise$. Add a RecomputeFieldValue.FieldOffset manual substitution for com.twitter.util.Promise$.
warning: unknown locality of class Lcom/twitter/io/Buf$HashingProcessor$1;, assuming class is not local. To remove the warning report an issue to the library or language author. The issue is caused by Lcom/twitter/io/Buf$HashingProcessor$1; which is not following the naming convention.
    analysis:  16,244.00 ms
error: unsupported features in 6 methods
Detailed message:
Error: Must not have a started Thread in the image heap.
Trace: 	object java.util.concurrent.ForkJoinPool$WorkQueue
	object java.util.concurrent.ForkJoinPool$WorkQueue[]
	object java.util.concurrent.ForkJoinPool
	object com.github.benmanes.caffeine.cache.LocalCacheFactory$WSLi
	object com.github.benmanes.caffeine.cache.BoundedLocalCache$BoundedLocalManualCache
	object com.twitter.finagle.stats.StatsReceiverWithCumulativeGauges$$anon$2$$anon$3
	object java.util.concurrent.ConcurrentHashMap$Node
	object java.util.concurrent.ConcurrentHashMap$Node[]
	object java.util.concurrent.ConcurrentHashMap
	object com.twitter.finagle.stats.MetricsStatsReceiver
	object com.twitter.finagle.stats.LoadedStatsReceiver$
	field com.twitter.finagle.stats.LoadedStatsReceiver$.MODULE$
Error: Must not have a started Thread in the image heap.
Trace: 	object java.util.concurrent.ForkJoinPool$WorkQueue
	object java.util.concurrent.ForkJoinPool$WorkQueue[]
	object java.util.concurrent.ForkJoinPool
	object com.github.benmanes.caffeine.cache.LocalCacheFactory$WSLi
	object com.github.benmanes.caffeine.cache.BoundedLocalCache$BoundedLocalManualCache
	object com.twitter.finagle.stats.StatsReceiverWithCumulativeGauges$$anon$2$$anon$3
	object java.util.concurrent.ConcurrentHashMap$Node
	object java.util.concurrent.ConcurrentHashMap$Node[]
	object java.util.concurrent.ConcurrentHashMap
	object com.twitter.finagle.stats.MetricsStatsReceiver
	object com.twitter.finagle.stats.LoadedStatsReceiver$
	field com.twitter.finagle.stats.LoadedStatsReceiver$.MODULE$
Error: com.oracle.graal.pointsto.constraints.UnsupportedFeatureException: Unsupported field java.lang.reflect.Proxy.proxyClassCache is reachable
To diagnose the issue, you can add the option -H:+ReportUnsupportedElementsAtRuntime. The unsupported element is then reported at run time when it is accessed the first time.
Trace:
	at parsing java.lang.reflect.Proxy.getProxyClass0(Proxy.java:419)
Call path from entry point to java.lang.reflect.Proxy.getProxyClass0(ClassLoader, Class[]):
	at java.lang.reflect.Proxy.getProxyClass0(Proxy.java:412)
	at java.lang.reflect.Proxy.newProxyInstance(Proxy.java:719)
	at com.twitter.util.SunSignalHandler.handle(Signal.scala:58)
	at com.twitter.util.HandleSignal$.$anonfun$apply$1(Signal.scala:85)
	at com.twitter.util.HandleSignal$.$anonfun$apply$1$adapted(Signal.scala:85)
	at com.twitter.util.HandleSignal$$$Lambda$828/1664204892.apply(Unknown Source)
	at scala.Option.foreach(Option.scala:257)
	at com.twitter.logging.FileHandler.publish(FileHandler.scala:283)
	at java.util.logging.Logger.log(Logger.java:738)
	at java.util.logging.Logger.doLog(Logger.java:765)
	at java.util.logging.Logger.log(Logger.java:876)
	at com.github.benmanes.caffeine.cache.BoundedLocalCache.lambda$notifyRemoval$1(BoundedLocalCache.java:288)
	at com.github.benmanes.caffeine.cache.BoundedLocalCache$$Lambda$857/1148776417.run(Unknown Source)
	at com.oracle.svm.core.JavaMainWrapper$JavaMainSupport.executeHooks(JavaMainWrapper.java:105)
	at com.oracle.svm.core.JavaMainWrapper$JavaMainSupport.executeStartupHooks(JavaMainWrapper.java:91)
	at com.oracle.svm.core.JavaMainWrapper.run(JavaMainWrapper.java:198)
	at Lcom/oracle/svm/core/code/CEntryPointCallStubs;.com_002eoracle_002esvm_002ecore_002eJavaMainWrapper_002erun_0028int_002corg_002egraalvm_002enativeimage_002ec_002etype_002eCCharPointerPointer_0029(generated:0)
Error: com.oracle.graal.pointsto.constraints.UnsupportedFeatureException: Unsupported field java.lang.reflect.Proxy.proxyClassCache is reachable
To diagnose the issue, you can add the option -H:+ReportUnsupportedElementsAtRuntime. The unsupported element is then reported at run time when it is accessed the first time.
Trace:
	at parsing java.lang.reflect.Proxy.isProxyClass(Proxy.java:791)
Call path from entry point to java.lang.reflect.Proxy.isProxyClass(Class):
	at java.lang.reflect.Proxy.isProxyClass(Proxy.java:791)
	at sun.reflect.misc.ReflectUtil.isNonPublicProxyClass(ReflectUtil.java:289)
	at java.lang.reflect.Proxy.checkNewProxyPermission(Proxy.java:757)
	at java.lang.reflect.Proxy.newProxyInstance(Proxy.java:726)
	at com.twitter.util.SunSignalHandler.handle(Signal.scala:58)
	at com.twitter.util.HandleSignal$.$anonfun$apply$1(Signal.scala:85)
	at com.twitter.util.HandleSignal$.$anonfun$apply$1$adapted(Signal.scala:85)
	at com.twitter.util.HandleSignal$$$Lambda$828/1664204892.apply(Unknown Source)
	at scala.Option.foreach(Option.scala:257)
	at com.twitter.logging.FileHandler.publish(FileHandler.scala:283)
	at java.util.logging.Logger.log(Logger.java:738)
	at java.util.logging.Logger.doLog(Logger.java:765)
	at java.util.logging.Logger.log(Logger.java:876)
	at com.github.benmanes.caffeine.cache.BoundedLocalCache.lambda$notifyRemoval$1(BoundedLocalCache.java:288)
	at com.github.benmanes.caffeine.cache.BoundedLocalCache$$Lambda$857/1148776417.run(Unknown Source)
	at com.oracle.svm.core.JavaMainWrapper$JavaMainSupport.executeHooks(JavaMainWrapper.java:105)
	at com.oracle.svm.core.JavaMainWrapper$JavaMainSupport.executeStartupHooks(JavaMainWrapper.java:91)
	at com.oracle.svm.core.JavaMainWrapper.run(JavaMainWrapper.java:198)
	at Lcom/oracle/svm/core/code/CEntryPointCallStubs;.com_002eoracle_002esvm_002ecore_002eJavaMainWrapper_002erun_0028int_002corg_002egraalvm_002enativeimage_002ec_002etype_002eCCharPointerPointer_0029(generated:0)
Error: com.oracle.graal.pointsto.constraints.UnsupportedFeatureException: Unsupported method java.lang.ClassLoader.getParent() is reachable: The declaring class of this element has been substituted, but this element is not present in the substitution class
To diagnose the issue, you can add the option -H:+ReportUnsupportedElementsAtRuntime. The unsupported element is then reported at run time when it is accessed the first time.
Trace:
	at parsing com.twitter.app.ClassPath.getEntries(ClassPath.scala:66)
Call path from entry point to com.twitter.app.ClassPath.getEntries(ClassLoader):
	at com.twitter.app.ClassPath.getEntries(ClassPath.scala:65)
	at com.twitter.app.ClassPath.browse(ClassPath.scala:57)
	at com.twitter.app.GlobalFlag$.getAll(GlobalFlag.scala:169)
	at com.twitter.app.GlobalFlag$.getAllOrEmptyArray(GlobalFlag.scala:146)
	at com.twitter.app.Flags.usage(Flags.scala:331)
	at com.twitter.app.App.$anonfun$main$1(App.scala:309)
	at com.twitter.app.App$$Lambda$691/320891981.apply(Unknown Source)
	at com.twitter.logging.Logger$.$anonfun$configure$1(Logger.scala:455)
	at com.twitter.logging.Logger$$$Lambda$760/2144147057.apply(Unknown Source)
	at scala.collection.immutable.List.foreach(List.scala:378)
	at com.twitter.util.Duration.toString(Duration.scala:338)
	at java.lang.String.valueOf(String.java:2994)
	at java.lang.StringBuilder.append(StringBuilder.java:131)
	at com.oracle.svm.core.amd64.AMD64CPUFeatureAccess.verifyHostSupportsArchitecture(AMD64CPUFeatureAccess.java:163)
	at com.oracle.svm.core.JavaMainWrapper.run(JavaMainWrapper.java:180)
	at Lcom/oracle/svm/core/code/CEntryPointCallStubs;.com_002eoracle_002esvm_002ecore_002eJavaMainWrapper_002erun_0028int_002corg_002egraalvm_002enativeimage_002ec_002etype_002eCCharPointerPointer_0029(generated:0)
Error: com.oracle.graal.pointsto.constraints.UnsupportedFeatureException: Unsupported method java.lang.ClassLoader.getParent() is reachable: The declaring class of this element has been substituted, but this element is not present in the substitution class
To diagnose the issue, you can add the option -H:+ReportUnsupportedElementsAtRuntime. The unsupported element is then reported at run time when it is accessed the first time.
Trace:
	at parsing sun.reflect.misc.ReflectUtil.isAncestor(ReflectUtil.java:207)
Call path from entry point to sun.reflect.misc.ReflectUtil.isAncestor(ClassLoader, ClassLoader):
	at sun.reflect.misc.ReflectUtil.isAncestor(ReflectUtil.java:205)
	at sun.reflect.misc.ReflectUtil.needsPackageAccessCheck(ReflectUtil.java:233)
	at sun.reflect.misc.ReflectUtil.checkProxyPackageAccess(ReflectUtil.java:269)
	at java.lang.reflect.Proxy.checkProxyAccess(Proxy.java:402)
	at java.lang.reflect.Proxy.newProxyInstance(Proxy.java:713)
	at com.twitter.util.SunSignalHandler.handle(Signal.scala:58)
	at com.twitter.util.HandleSignal$.$anonfun$apply$1(Signal.scala:85)
	at com.twitter.util.HandleSignal$.$anonfun$apply$1$adapted(Signal.scala:85)
	at com.twitter.util.HandleSignal$$$Lambda$828/1664204892.apply(Unknown Source)
	at scala.Option.foreach(Option.scala:257)
	at com.twitter.logging.FileHandler.publish(FileHandler.scala:283)
	at java.util.logging.Logger.log(Logger.java:738)
	at java.util.logging.Logger.doLog(Logger.java:765)
	at java.util.logging.Logger.log(Logger.java:876)
	at com.github.benmanes.caffeine.cache.BoundedLocalCache.lambda$notifyRemoval$1(BoundedLocalCache.java:288)
	at com.github.benmanes.caffeine.cache.BoundedLocalCache$$Lambda$857/1148776417.run(Unknown Source)
	at com.oracle.svm.core.JavaMainWrapper$JavaMainSupport.executeHooks(JavaMainWrapper.java:105)
	at com.oracle.svm.core.JavaMainWrapper$JavaMainSupport.executeStartupHooks(JavaMainWrapper.java:91)
	at com.oracle.svm.core.JavaMainWrapper.run(JavaMainWrapper.java:198)
	at Lcom/oracle/svm/core/code/CEntryPointCallStubs;.com_002eoracle_002esvm_002ecore_002eJavaMainWrapper_002erun_0028int_002corg_002egraalvm_002enativeimage_002ec_002etype_002eCCharPointerPointer_0029(generated:0)

Error: Image building with exit status 1

[wmorgan]

@zackangelo Thanks for taking a stab. Some other folks have been looking into this too. Would you be interested in joining our nascent working group?