To run the benchmark suite locally you have the additional requirement of having python installed. Once that is done follow these steps:

1. Go to the PhotoshopBenchmark folder

   $ cd <dir_to_photoshop_api>/PhotoshopBenchmark

2. Create a venv and install the requirements.txt (these instructions are for windows)

   $ py -m venv .venv
   
   $ .venv/Scripts/activate.bat
   
   $ py -m pip install -r requirements.txt
   

You can now choose whether to only run the PhotoshopAPI benchmarks or also the Photoshop benchmarks after which you can Generate Graphs

Due to the benchmark files being many GB large they are not stored in the repository themselves but rather hosted elsewhere

To run the PhotoshopAPI benchmarks please first download the benchmark files from Google Drive (Thanks to Glacious for graciously providing some of the test files)

Move the folders into the PhotoshopBenchmark/documents/ folder. You should be left with the following folder structure:

PhotoshopAPI/
    PhotoshopBenchmark/
        documents/
            write/
                # Test .psd and .psb are here
            read/
                # Empty folder which will be populated on write

To run the PhotoshopAPI benchmarks we must first build the PhotoshopBenchmark executable. To do that follow the build steps outlined on ReadTheDocs and make sure the CMake flag PSAPI_BUILD_BENCHMARKS is enabled as well as ensuring the build mode is release.

Once that is done you should find a PhotoshopBenchmark executable in PhotoshopAPI/bin-int/PhotoshopAPI/x64-release/PhotoshopBenchmark which needs to be copied over to PhotoshopAPI/PhotoshopBenchmark.

Executing it will create a benchmarkStatisticsPSAPI.txt file containing the outputs for our plot generation. Please note that these benchmarks can take a significant amount of time.

To run the Photoshop benchmarks you must have Photoshop installed unlike when running the PhotoshopAPI benchmarks. You then call the src/benchmark_photoshop.py script which will reopen Photoshop between runs as it otherwise stores some document information on disk which will make the run times non-repeatable.

The script can be called like this:

    $ py -u src/benchmark_photoshop.py

PhotoshopAPI uses matplotlib to create graphs of our benchmarks which it stores into the PhotoshopAPI/PhotoshopBenchmark/plots directory.

To invoke the python script that creates the graphs you need to have two *.txt files in the PhotoshopBenchmark/ dir. You need the benchmarkStatisticsPhotoshop.txt generated from the Photoshop benchmarks as well as the benchmarkStatisticsPSAPI.txt generated by the PhotoshopAPI benchmarks.

If you have both of these you call the src/create_plots.py script

    $ py -u src/create_plots.py

which will generate the appropriately named plots ready to be copied over to the docs/ folder