Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update benchmark scripts #213

Merged
merged 53 commits into from
Jul 26, 2021
Merged

Update benchmark scripts #213

merged 53 commits into from
Jul 26, 2021

Conversation

denisalevi
Copy link
Member

This PR should

  • add scripts that allow to easily run our benchmark scripts on our cluster
  • fix any bugs found on the way

I opened the PR mainly for code review purposes.

SudeshnaBora and others added 30 commits June 16, 2021 12:36
@SudeshnaBora
Add running benchmark in remote script
9a819ec
@SudeshnaBora
Improve functionality of calling benchmark suite
1b6cabb
@SudeshnaBora
Rename file
338d4cd
@SudeshnaBora
Add new attribute in the configuration file
4551301
@denisalevi
Run benchmarks on RTX2080 by default
450215c
@denisalevi
Move benchmark scripts into tools folder
e89869c
@denisalevi
Sort tools scripts into folders
5feced4
@denisalevi
Make _on_headnode.sh work with any script
dee92e1
@denisalevi
Fix relative paths in tools scripts
988296b
@denisalevi
Exclude dev folder from copied git repository in benchmarks
b04ad50
@denisalevi
Add arguments for results dir to benchmark script
c6844a4
@denisalevi
Update gitignore with new benchmark folders
d9d0559
@denisalevi
Don't fail benchmark script without brian2genn installed
5849743
@denisalevi
Update tools README
21d1976
@denisalevi
Exclude worktrees folder from test suite sync
8c0ebe2
@denisalevi
Fix bug in test suite remote script
0f18d1e
@SudeshnaBora
Create/Refactor benchmarking scripts
7fc4630
@denisalevi
Make benchmark README image links relative
b490a71
@denisalevi
Modify benchmark configuration for first run
267bc76
@denisalevi
Adapt benchmark scripts
0b73942
@denisalevi
Update wrong brian2 and brian2genn submodule commits
7e59828 
It looks like I updated brian2 and brian2genn in order to use
brian2genn's kernel_timing functions for profiling. I added custom code
to brian2genn, which I commited in the `brian2genn.diff` file, but it
seems I didn't update the brian2 and brian2genn submodules. I now did
that based on the `git.log` records from our last benchmarks in 11/2018.
@denisalevi
Update benchmark scripts
6e6d74e 
Clean up previous commit and add an option to not delete the remote
brian2cuda repository after benchmark or test suite run (for debugging).
@denisalevi
Print if GENN_PATH or CUDA_PATH are modified
41605d7
@denisalevi
Add script to run benchmarks locally
7dd0f82
@denisalevi
Support git worktrees in remote benchmark script
174a996 
The brian2cuda repository is now pulled form GitHub on the remote and
updated via the local repository (instead of using rsync for copy). This
should work even for slow local internet connection.

Uncommited changes will not be benchmarked now.
@denisalevi
Acknowledge CUDA_DEVICE_QUERY during GPU selection
d17d3db 
We use `nvidia-smi -L` to detect all available GPUs. `nvidia-smi`
displays all GPUs independent of `CUDA_DEVICE_QUERY`, hence setting it
had not effect. Now, `CUDA_DEVICE_QUERY=1` will detect GPU 1 as GPU 0.
@denisalevi
Remove whitespace
bdad904
@denisalevi
Update gpu_id prefs documentation
81aafbc 
This is not implemented yet, but shows how it should work.
@denisalevi
Fix CUDA_VISIBLE_DEVICES behavior
7f96d86 
CUDA_VISIBLE_DEVICES will now precede any other options. That means if
`gpu_id` is set as `prefs`, it will choose from the visible devices.
@denisalevi
Rename python benchmark script to run_benchmark_suite.py
abff3e7
denisalevi and others added 23 commits July 14, 2021 18:49
@denisalevi
Make python benchmark script usable directly
6f05116
@denisalevi
Make benchmark shell scripts executable
7bc801b
@denisalevi
Update README for remote run scripts
3d7b318
@denisalevi
Add README to test_suite scripts folder
8e51ff1
@denisalevi
Update benchmarking gitignore
d378e38
@denisalevi
Update main gitignore
d908837
@denisalevi
Add README for benchmark scripts
4f15e3f
@denisalevi
Update tools README
465172f
@denisalevi
Minor benchmark fixes
d6574f9
@denisalevi
Remove colons from synced benchmark repo names
ac4fa4d 
Synchronized brian2cuda repositories included colons in their folder
names from the appended time stamps. This broke setting `PATH` for genn
binaries correctly, since colons are interpreted as separators in
`PATH`.
@denisalevi
Don't source for test suite
a02baa3
@denisalevi
Add option to benchmark script for copying tools dir
9554f7d
@denisalevi
Fix DynamicConfigCreator for repo without stash
f833352
@denisalevi
Fix setting GeNN environment variables in benchmarks
72adfd9 
Don't pipe after exporting environment variables, it will not export
them since an intermediate shell is created
@denisalevi
Fix typo and remove debug prints
c1a86a8
@denisalevi
Set GeNN bin folder at start of PATH
39be335
@denisalevi
Silence brian2cuda diagnostic logging in benchmarks
df19068
@denisalevi
Update remote tools README
c052f38
@denisalevi
Fix remote benchmark README creation
a6a539f
@denisalevi
Add script to connect to grip on remote
f4001e3
@denisalevi
Use 24 threads for openmp benchmarks and update bench config
ab44d09
@denisalevi
Make grip port configurable in script
f0a528b
@SudeshnaBora
Addlatest benchmarking results
21176c7
@denisalevi denisalevi merged commit d9997f0 into master Jul 26, 2021
@denisalevi denisalevi deleted the benchmarking branch August 10, 2021 12:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@denisalevi @SudeshnaBora