memoryFootprint reports negative memory usage

[lJoublanc]

Can you confirm whether this is supposed to happen?

[info] ::Benchmark Chunk.Columns.fold::
[info] cores: 4
[info] hostname: Luciano
[info] name: Java HotSpot(TM) 64-Bit Server VM
[info] osArch: amd64
[info] osName: Windows 7
[info] vendor: Oracle Corporation
[info] version: 25.161-b12
[info] Parameters(numRows -> 1): -30.92 kB
[info] Parameters(numRows -> 10000): -14.672 kB
[info]
[info] ::Benchmark Chunk.Columns.mapChunk::
[info] cores: 4
[info] hostname: Luciano
[info] name: Java HotSpot(TM) 64-Bit Server VM
[info] osArch: amd64
[info] osName: Windows 7
[info] vendor: Oracle Corporation
[info] version: 25.161-b12
[info] Parameters(numRows -> 1): 2850.856 kB
[info] Parameters(numRows -> 10000): -14.432 kB

If you can confirm this is not expected behaviour, I can see if I can publish the tests causing this.

[kyledewey]

Possible duplicate of #24. I unfortunately can't be very helpful; I'm seeing the same issue and I similarly don't know the root cause. Not all benchmarks on my end are seeing negatives, but the negatives make me hesitant to trust the other results.

[bertrandkang]

The MemoryFootprint measurer uses the totalMemory and freeMemory statistics from the JVM's Runtime class (https://github.com/scalameter/scalameter/blob/master/scalameter-core/src/main/scala/org/scalameter/Measurer.scala#L412). Footprint is measured by running these before and after your snippet, and comparing. Since the JVM tends to GC unpredictably, and there are other threads allocating and deallocating memory, these methods can somewhat noisy. For this reason, it is important to repeat the measurement multiple times, and use relatively large objects.

Are you returning the object whose size you want to measure from the test? Can you increase the number of execution runs, or the size of the measured object?

Also, it's generally helpful to post the complete snippet where this happens.

[kyledewey]

For my specific case, I'm interested in peak memory usage during execution. It's acceptable if this can be measured only after GC occurs. The main issue with measuring at the end in my case is that my application allocates memory frequently and creates lots of garbage in the process, and memory usage is expected to be at its peak about halfway through execution.

Is there a way to gather this sort of peak memory usage with scalameter?

[axel22]

Hi, Yes, there is a way to do it, but you would have to define your own `Measurer` subclass. You would have to implement its `measure` method to, for example: 1. Run GC and immediately call `totalMemory - freeMemory` to determine the baseline footprint. 2. Start a new thread that will repetitively sample the `totalMemory - freeMemory`. 3. Run the benchmark snippet. 4. Tell the thread that was doing the sampling to stop, and collect its readings. 5. Find the maximum value among those readings. Then, you should use your custom measurer to when overriding the `measurer` method of the benchmark snippet.

…

On Mon, Jun 18, 2018 at 11:57:30AM -0700, Kyle Dewey wrote: For my specific case, I'm interested in peak memory usage during execution. It's acceptable if this can be measured only after GC occurs. The main issue with measuring at the end in my case is that my application allocates memory frequently and creates lots of garbage in the process, and memory usage is expected to be at its peak about halfway through execution. Is there a way to gather this sort of peak memory usage with scalameter? - You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.