Windows 64-bit crashes in pluginval

[sophiapoirier]

According to this forum message:
https://www.kvraudio.com/forum/viewtopic.php?p=8195304#p8195304
all of our current Windows 64-bit builds crash the diagnostic tool pluginval. I have used this for testing in the past and the AUs all pass fine on the maximum strictness level of 10, but apparently at level 8 our Windows VSTs crash. Might be worth investigating, as it may perhaps be triggering crashing that could occur under normal usage? (The forum poster mentions some regular usage crashing as well.)

The tool should hopefully be simple to build from source (it was for Mac) and the usage then would be:
pluginval --strictness-level 10 --validate path/to/plugin.dll

[tom7]

I don't think I can compile this easily since it seems to want Visual Studio, but I got the binaries. I can only get it to validate in-process, but it works at strictness 8 for dfx-bufferoverride, for one:

$ ./pluginval.exe --strictness-level 10 --validate-in-process --validate ../buf
feroverride/win32/dfx-bufferoverride-64.dll
pluginval v0.2.7 - JUCE v5.4.7
Started validating: ../bufferoverride/win32/dfx-bufferoverride-64.dll
Random seed: 0x6014a71
Validation started: 13 Sep 2021 5:53:10am

Strictness level: 10
-----------------------------------------------------------------
Starting test: pluginval / Scan for known types: ../bufferoverride/win32/dfx-bufferoverride-64.dll...
Num types found: 1

Testing plugin: VST-dfx-bufferoverride-64-95ba40e7-62756666
Destroy FX: dfx-bufferoverride-64 v2.7.0.0
All tests completed successfully
-----------------------------------------------------------------
Starting test: pluginval / Open plugin (cold)...

Time taken to open plugin (cold): 549 ms
All tests completed successfully
-----------------------------------------------------------------
Starting test: pluginval / Open plugin (warm)...

Time taken to open plugin (warm): 6 ms
Running tests 1 times
All tests completed successfully
-----------------------------------------------------------------
Starting test: pluginval / Plugin info...

Plugin name: Super Destroy FX bipolar plugin pack
Alternative names: Super Destroy FX bipolar plugin pack
SupportsDoublePrecision: no
Reported latency: 0
Reported taillength: 5.71429

Time taken to run test: 0
All tests completed successfully
-----------------------------------------------------------------
Starting test: pluginval / Plugin programs...
Num programs: 16
All program names checked

Changing program

Time taken to run test: 0
All tests completed successfully
-----------------------------------------------------------------
Starting test: pluginval / Editor...

Time taken to open editor (cold): 234 ms
Time taken to open editor (warm): 213 ms

Time taken to run test: 462 ms
All tests completed successfully
-----------------------------------------------------------------
Starting test: pluginval / Open editor whilst processing...

Time taken to run test: 221 ms
All tests completed successfully
-----------------------------------------------------------------
Starting test: pluginval / Audio processing...
Testing with sample rate [44100] and block size [64]
Testing with sample rate [44100] and block size [128]
Testing with sample rate [44100] and block size [256]
Testing with sample rate [44100] and block size [512]
Testing with sample rate [44100] and block size [1024]
Testing with sample rate [48000] and block size [64]
Testing with sample rate [48000] and block size [128]
Testing with sample rate [48000] and block size [256]
Testing with sample rate [48000] and block size [512]
Testing with sample rate [48000] and block size [1024]
Testing with sample rate [96000] and block size [64]
Testing with sample rate [96000] and block size [128]
Testing with sample rate [96000] and block size [256]
Testing with sample rate [96000] and block size [512]
Testing with sample rate [96000] and block size [1024]

Time taken to run test: 4 ms
All tests completed successfully
-----------------------------------------------------------------
Starting test: pluginval / Plugin state...

Time taken to run test: 0
All tests completed successfully
-----------------------------------------------------------------
Starting test: pluginval / Plugin state restoration...

Time taken to run test: 0
All tests completed successfully
-----------------------------------------------------------------
Starting test: pluginval / Automation...
Testing with sample rate [44100] and block size [64] and sub-block size [32]
Testing with sample rate [44100] and block size [128] and sub-block size [32]
Testing with sample rate [44100] and block size [256] and sub-block size [32]
Testing with sample rate [44100] and block size [512] and sub-block size [32]
Testing with sample rate [44100] and block size [1024] and sub-block size [32]
Testing with sample rate [48000] and block size [64] and sub-block size [32]
Testing with sample rate [48000] and block size [128] and sub-block size [32]
Testing with sample rate [48000] and block size [256] and sub-block size [32]
Testing with sample rate [48000] and block size [512] and sub-block size [32]
Testing with sample rate [48000] and block size [1024] and sub-block size [32]
Testing with sample rate [96000] and block size [64] and sub-block size [32]
Testing with sample rate [96000] and block size [128] and sub-block size [32]
Testing with sample rate [96000] and block size [256] and sub-block size [32]
Testing with sample rate [96000] and block size [512] and sub-block size [32]
Testing with sample rate [96000] and block size [1024] and sub-block size [32]

Time taken to run test: 2 ms
All tests completed successfully
-----------------------------------------------------------------
Starting test: pluginval / Editor Automation...

Time taken to run test: 11 secs
All tests completed successfully
-----------------------------------------------------------------
Starting test: pluginval / Automatable Parameters...

Time taken to run test: 0
All tests completed successfully
-----------------------------------------------------------------
Starting test: pluginval / Parameters...

Time taken to run test: 0
All tests completed successfully
-----------------------------------------------------------------
Starting test: pluginval / Background thread state...

Time taken to run test: 2 secs
All tests completed successfully
-----------------------------------------------------------------
Starting test: pluginval / Parameter thread safety...

Time taken to run test: 4 ms
All tests completed successfully
-----------------------------------------------------------------
Starting test: pluginval / Basic bus...
All tests completed successfully
-----------------------------------------------------------------
Starting test: pluginval / Listing available buses...
Inputs:
        Named layouts: Mono, Stereo
        Discrete layouts: Discrete #1, Discrete #2
Outputs:
        Named layouts: Mono, Stereo
        Discrete layouts: Discrete #1, Discrete #2
Main bus num input channels: 2
Main bus num output channels: 2
All tests completed successfully
-----------------------------------------------------------------
Starting test: pluginval / Enabling all buses...
All tests completed successfully
-----------------------------------------------------------------
Starting test: pluginval / Disabling non-main busses...
All tests completed successfully
-----------------------------------------------------------------
Starting test: pluginval / Restoring default layout...
Main bus num input channels: 2
Main bus num output channels: 2

Time taken to run test: 1 ms
All tests completed successfully
-----------------------------------------------------------------
Starting test: pluginval / Editor stress...
Testing opening Editor with released processing
Testing opening Editor with 0 sample rate and block size

Time taken to run test: 444 ms
All tests completed successfully
-----------------------------------------------------------------
Starting test: pluginval / Editor DPI Awareness...
Testing opening Editor with DPI Awareness disabled

Time taken to run test: 222 ms
All tests completed successfully
-----------------------------------------------------------------
Starting test: pluginval / Allocations during process...
Testing with sample rate [44100] and block size [64]
Testing with sample rate [44100] and block size [128]
Testing with sample rate [44100] and block size [256]
Testing with sample rate [44100] and block size [512]
Testing with sample rate [44100] and block size [1024]
Testing with sample rate [48000] and block size [64]
Testing with sample rate [48000] and block size [128]
Testing with sample rate [48000] and block size [256]
Testing with sample rate [48000] and block size [512]
Testing with sample rate [48000] and block size [1024]
Testing with sample rate [96000] and block size [64]
Testing with sample rate [96000] and block size [128]
Testing with sample rate [96000] and block size [256]
Testing with sample rate [96000] and block size [512]
Testing with sample rate [96000] and block size [1024]

Time taken to run test: 8 ms
All tests completed successfully
-----------------------------------------------------------------
Starting test: pluginval / Process called with a larger than prepared block size...
INFO: Skipping test for plugin format

Time taken to run test: 1 ms
All tests completed successfully
-----------------------------------------------------------------
Starting test: pluginval / Fuzz parameters...

Time taken to run test: 0

Time taken to run all tests: 15 secs
All tests completed successfully

Finished validating: ../bufferoverride/win32/dfx-bufferoverride-64.dll
ALL TESTS PASSED

I did see one crash on a dfx-geometer that I had lying around, but I can't reproduce when building from head, so perhaps it was that? Am I remembering that you fixed some fishy stuff in there recently? Or maybe there are some unlucky race conditions.

[sophiapoirier]

Okay great! Yeah perhaps it was an issue of older state of the code, or something wrong in the person's development environment. It sounded to me like an "every plugin every time" thing, so if you got zero failures on any of them, I think we can not worry about this?

[tom7]

Yeah, I think this is "can't reproduce," although it's nice to have this tool around for future iterations!