Errors when using SeparateJvmsExecutor

[ColOfAbRiX]

I'm using Scalameter in a personal project with these versions:

• Scala 2.13.2
• Scalameter 0.20
• SBT 1.4.6

When I use the executor SeparateJvmsExecutor I run into a series of problems. The first thing that happens is this exception:

Exception in thread "main" java.lang.NoSuchMethodError: scala.Product.$init$(Lscala/Product;)V
        at org.scalameter.execution.SeparateJvmFailure.<init>(SeparateJvmFailure.scala:3)
        at org.scalameter.execution.Main$.saveFailure(Main.scala:50)
        at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2960)
        at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1540)
        at java.io.ObjectInputStream.readObject(ObjectInputStream.java:431)
        at org.scalameter.execution.JvmRunner.readOutput(JvmRunner.scala:74)
        at org.scalameter.execution.JvmRunner.run(JvmRunner.scala:22)

This is the custom benchmark I'm using:

import org.scalameter.api._
import org.scalameter.picklers.Implicits._

abstract class FigletBenchmark extends Bench[Double] {
  import org.scalameter.reporting._
  import org.scalameter._

  def executor: Executor[Double] = new execution.SeparateJvmsExecutor(warmer, aggregator, measurer)

  private def warmer: Warmer = new Warmer.Default

  private def aggregator: Aggregator[Double] = Aggregator.average

  def measurer: Measurer[Double] =
    new Measurer.IgnoringGC with Measurer.PeriodicReinstantiation[Double]
    with Measurer.OutlierElimination[Double] with Measurer.RelativeNoise {
      def numeric: Numeric[Double] = implicitly[Numeric[Double]]
    }

  def persistor: Persistor =
    new persistence.GZIPJSONSerializationPersistor

  override def reporter: Reporter[Double] =
    Reporter.Composite(
      new RegressionReporter(tester, historian),
      HtmlReporter(embedDsv = true),
    )

  private def tester: RegressionReporter.Tester = RegressionReporter.Tester.Accepter()

  private def historian: RegressionReporter.Historian = RegressionReporter.Historian.ExponentialBackoff()
}

Digging into the issue I found that JvmRunner includes SBT jars in the classpath, and that creates conflicts.

[ColOfAbRiX]

See ColOfAbRiX@08cb917 for a possible solution. I'm just removing any classpath that refers to SBT

[ColOfAbRiX]

After that issue I started receving another exception:

java.lang.ClassNotFoundException: org.scalameter.Parameters
        at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
        at java.lang.Class.forName0(Native Method)
        at java.lang.Class.forName(Class.java:348)
        at java.io.ObjectInputStream.resolveClass(ObjectInputStream.java:686)
        at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1868)
        at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1751)
        at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2042)
        at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1573)
        at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2287)
        at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2211)
        at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2069)
        at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1573)
        at java.io.ObjectInputStream.readObject(ObjectInputStream.java:431)
        at scala.collection.generic.DefaultSerializationProxy.readObject(DefaultSerializationProxy.scala:54)
        at sun.reflect.GeneratedMethodAccessor46.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1170)
        at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2178)
        at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2069)
        at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1573)
        at java.io.ObjectInputStream.readObject(ObjectInputStream.java:431)
        at org.scalameter.execution.JvmRunner.readOutput(JvmRunner.scala:74)
        at org.scalameter.execution.JvmRunner.run(JvmRunner.scala:22)

and with a bit of digging it's a known issue with Scala serialization as discussed in this thread scala/bug#9237

I added a new object to deserialize the data in this commit ColOfAbRiX@99d54ae

These are quick solutions to the problems and I might not have considered everything. Let me know if you want a PR for those changes or if you think there's more work to do.

[ColOfAbRiX]

Unfortunately that's not the end of it.

While the LoggingReporter() reporter works, it seems that using as I do in my code:

 Reporter.Composite(
   new RegressionReporter(tester, historian),
   HtmlReporter(embedDsv = true),
)

creates more problems with different exceptions.

[froth]

Same here :(

[MercurieVV]

My workaround for this is following.
I already created sbt config Benchmark for this.
In my module I specify classloader for this config inConfig(Benchmark)(classLoaderLayeringStrategy := ClassLoaderLayeringStrategy.Flat)
Maybe other strategies will work too. I didnt try to get deeper and check