Try fixing AMDGPU test stalling #716

avik-pal · 2024-06-19T00:44:23Z

No description provided.

codecov · 2024-06-19T00:48:10Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 87.14%. Comparing base (237831d) to head (ad7acc0).

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #716      +/-   ##
==========================================
- Coverage   87.19%   87.14%   -0.05%     
==========================================
  Files          50       50              
  Lines        2522     2521       -1     
==========================================
- Hits         2199     2197       -2     
- Misses        323      324       +1

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

github-actions

Benchmark Results

Benchmark suite	Current: `ad7acc0`	Previous: `237831d`	Ratio
`Dense(2 => 2)/cpu/reverse/ReverseDiff (compiled)/(2, 128)`	`3882.25` ns	`3694.375` ns	`1.05`
`Dense(2 => 2)/cpu/reverse/Zygote/(2, 128)`	`7277` ns	`7175.4` ns	`1.01`
`Dense(2 => 2)/cpu/reverse/Tracker/(2, 128)`	`20929` ns	`21109` ns	`0.99`
`Dense(2 => 2)/cpu/reverse/ReverseDiff/(2, 128)`	`9932.6` ns	`9923.5` ns	`1.00`
`Dense(2 => 2)/cpu/reverse/Flux/(2, 128)`	`8857` ns	`8936.8` ns	`0.99`
`Dense(2 => 2)/cpu/reverse/SimpleChains/(2, 128)`	`4462.125` ns	`4492.25` ns	`0.99`
`Dense(2 => 2)/cpu/reverse/Enzyme/(2, 128)`	`1165.7724137931034` ns	`1164.4202898550725` ns	`1.00`
`Dense(2 => 2)/cpu/forward/NamedTuple/(2, 128)`	`1172.485294117647` ns	`1112.5704225352113` ns	`1.05`
`Dense(2 => 2)/cpu/forward/ComponentArray/(2, 128)`	`1173.3525179856115` ns	`1178` ns	`1.00`
`Dense(2 => 2)/cpu/forward/Flux/(2, 128)`	`1782.1803278688524` ns	`1797.4705882352941` ns	`0.99`
`Dense(2 => 2)/cpu/forward/SimpleChains/(2, 128)`	`180.30921985815604` ns	`180.1279554937413` ns	`1.00`
`Dense(20 => 20)/cpu/reverse/ReverseDiff (compiled)/(20, 128)`	`17332` ns	`17353` ns	`1.00`
`Dense(20 => 20)/cpu/reverse/Zygote/(20, 128)`	`17142.5` ns	`17052` ns	`1.01`
`Dense(20 => 20)/cpu/reverse/Tracker/(20, 128)`	`37580` ns	`37640` ns	`1.00`
`Dense(20 => 20)/cpu/reverse/ReverseDiff/(20, 128)`	`29386` ns	`29785` ns	`0.99`
`Dense(20 => 20)/cpu/reverse/Flux/(20, 128)`	`21500` ns	`21450` ns	`1.00`
`Dense(20 => 20)/cpu/reverse/SimpleChains/(20, 128)`	`17403` ns	`17402` ns	`1.00`
`Dense(20 => 20)/cpu/reverse/Enzyme/(20, 128)`	`4356.714285714285` ns	`4325.142857142857` ns	`1.01`
`Dense(20 => 20)/cpu/forward/NamedTuple/(20, 128)`	`3882.25` ns	`3876` ns	`1.00`
`Dense(20 => 20)/cpu/forward/ComponentArray/(20, 128)`	`3963.75` ns	`3953.625` ns	`1.00`
`Dense(20 => 20)/cpu/forward/Flux/(20, 128)`	`4955` ns	`4953.428571428572` ns	`1.00`
`Dense(20 => 20)/cpu/forward/SimpleChains/(20, 128)`	`1655.1` ns	`1652.1` ns	`1.00`
`Conv((3, 3), 3 => 3)/cpu/reverse/ReverseDiff (compiled)/(64, 64, 3, 128)`	`40614742` ns	`47320777` ns	`0.86`
`Conv((3, 3), 3 => 3)/cpu/reverse/Zygote/(64, 64, 3, 128)`	`58011130` ns	`58305356` ns	`0.99`
`Conv((3, 3), 3 => 3)/cpu/reverse/Tracker/(64, 64, 3, 128)`	`79578158` ns	`102789420` ns	`0.77`
`Conv((3, 3), 3 => 3)/cpu/reverse/ReverseDiff/(64, 64, 3, 128)`	`91908921.5` ns	`95601238` ns	`0.96`
`Conv((3, 3), 3 => 3)/cpu/reverse/Flux/(64, 64, 3, 128)`	`78482625` ns	`78618619` ns	`1.00`
`Conv((3, 3), 3 => 3)/cpu/reverse/SimpleChains/(64, 64, 3, 128)`	`11698721` ns	`11718436` ns	`1.00`
`Conv((3, 3), 3 => 3)/cpu/reverse/Enzyme/(64, 64, 3, 128)`	`17817403.5` ns	`17850771.5` ns	`1.00`
`Conv((3, 3), 3 => 3)/cpu/forward/NamedTuple/(64, 64, 3, 128)`	`7022793.5` ns	`7036938` ns	`1.00`
`Conv((3, 3), 3 => 3)/cpu/forward/ComponentArray/(64, 64, 3, 128)`	`6989877` ns	`7001847` ns	`1.00`
`Conv((3, 3), 3 => 3)/cpu/forward/Flux/(64, 64, 3, 128)`	`12165840` ns	`11538399` ns	`1.05`
`Conv((3, 3), 3 => 3)/cpu/forward/SimpleChains/(64, 64, 3, 128)`	`6382128` ns	`6393151.5` ns	`1.00`
`vgg16/cpu/reverse/Zygote/(32, 32, 3, 16)`	`735127549` ns	`751448443` ns	`0.98`
`vgg16/cpu/reverse/Zygote/(32, 32, 3, 64)`	`2581803973` ns	`2573472812` ns	`1.00`
`vgg16/cpu/reverse/Zygote/(32, 32, 3, 2)`	`146214926` ns	`144720323` ns	`1.01`
`vgg16/cpu/reverse/Tracker/(32, 32, 3, 16)`	`915296738.5` ns	`968831713.5` ns	`0.94`
`vgg16/cpu/reverse/Tracker/(32, 32, 3, 64)`	`3192183030` ns	`3278943882` ns	`0.97`
`vgg16/cpu/reverse/Tracker/(32, 32, 3, 2)`	`238000000` ns	`234124583` ns	`1.02`
`vgg16/cpu/reverse/Flux/(32, 32, 3, 16)`	`755053591.5` ns	`746070446` ns	`1.01`
`vgg16/cpu/reverse/Flux/(32, 32, 3, 64)`	`2600372472` ns	`3009137266` ns	`0.86`
`vgg16/cpu/reverse/Flux/(32, 32, 3, 2)`	`148514039` ns	`132644713.5` ns	`1.12`
`vgg16/cpu/forward/NamedTuple/(32, 32, 3, 16)`	`174470555` ns	`174325259` ns	`1.00`
`vgg16/cpu/forward/NamedTuple/(32, 32, 3, 64)`	`651616997` ns	`647357046.5` ns	`1.01`
`vgg16/cpu/forward/NamedTuple/(32, 32, 3, 2)`	`34737511` ns	`34732801` ns	`1.00`
`vgg16/cpu/forward/ComponentArray/(32, 32, 3, 16)`	`165572691` ns	`164171075.5` ns	`1.01`
`vgg16/cpu/forward/ComponentArray/(32, 32, 3, 64)`	`633283440` ns	`641446171` ns	`0.99`
`vgg16/cpu/forward/ComponentArray/(32, 32, 3, 2)`	`30671671` ns	`30107004` ns	`1.02`
`vgg16/cpu/forward/Flux/(32, 32, 3, 16)`	`218411469` ns	`189802799.5` ns	`1.15`
`vgg16/cpu/forward/Flux/(32, 32, 3, 64)`	`763518919` ns	`799823428` ns	`0.95`
`vgg16/cpu/forward/Flux/(32, 32, 3, 2)`	`36550717` ns	`38276609` ns	`0.95`
`Conv((3, 3), 64 => 64)/cpu/reverse/ReverseDiff (compiled)/(64, 64, 64, 128)`	`1328700543.5` ns	`1306917435` ns	`1.02`
`Conv((3, 3), 64 => 64)/cpu/reverse/Zygote/(64, 64, 64, 128)`	`1885142073` ns	`1880303414` ns	`1.00`
`Conv((3, 3), 64 => 64)/cpu/reverse/Tracker/(64, 64, 64, 128)`	`2606608638` ns	`2465824739` ns	`1.06`
`Conv((3, 3), 64 => 64)/cpu/reverse/ReverseDiff/(64, 64, 64, 128)`	`2612304354` ns	`2587857217` ns	`1.01`
`Conv((3, 3), 64 => 64)/cpu/reverse/Flux/(64, 64, 64, 128)`	`1994257949.5` ns	`1920389453.5` ns	`1.04`
`Conv((3, 3), 64 => 64)/cpu/reverse/Enzyme/(64, 64, 64, 128)`	`560219293` ns	`561226426` ns	`1.00`
`Conv((3, 3), 64 => 64)/cpu/forward/NamedTuple/(64, 64, 64, 128)`	`322176825` ns	`325726548` ns	`0.99`
`Conv((3, 3), 64 => 64)/cpu/forward/ComponentArray/(64, 64, 64, 128)`	`318316689` ns	`323189696` ns	`0.98`
`Conv((3, 3), 64 => 64)/cpu/forward/Flux/(64, 64, 64, 128)`	`367559067` ns	`472300185.5` ns	`0.78`
`Conv((3, 3), 1 => 1)/cpu/reverse/ReverseDiff (compiled)/(64, 64, 1, 128)`	`11986658` ns	`11879578` ns	`1.01`
`Conv((3, 3), 1 => 1)/cpu/reverse/Zygote/(64, 64, 1, 128)`	`18023654.5` ns	`18066903` ns	`1.00`
`Conv((3, 3), 1 => 1)/cpu/reverse/Tracker/(64, 64, 1, 128)`	`19486562.5` ns	`19358439.5` ns	`1.01`
`Conv((3, 3), 1 => 1)/cpu/reverse/ReverseDiff/(64, 64, 1, 128)`	`24118039` ns	`24037285` ns	`1.00`
`Conv((3, 3), 1 => 1)/cpu/reverse/Flux/(64, 64, 1, 128)`	`18089883.5` ns	`18030067` ns	`1.00`
`Conv((3, 3), 1 => 1)/cpu/reverse/SimpleChains/(64, 64, 1, 128)`	`1173517` ns	`1161439` ns	`1.01`
`Conv((3, 3), 1 => 1)/cpu/reverse/Enzyme/(64, 64, 1, 128)`	`5850901` ns	`5877613` ns	`1.00`
`Conv((3, 3), 1 => 1)/cpu/forward/NamedTuple/(64, 64, 1, 128)`	`2060789` ns	`2061078` ns	`1.00`
`Conv((3, 3), 1 => 1)/cpu/forward/ComponentArray/(64, 64, 1, 128)`	`2037134.5` ns	`2052642` ns	`0.99`
`Conv((3, 3), 1 => 1)/cpu/forward/Flux/(64, 64, 1, 128)`	`2086456` ns	`2085073` ns	`1.00`
`Conv((3, 3), 1 => 1)/cpu/forward/SimpleChains/(64, 64, 1, 128)`	`203791` ns	`207838` ns	`0.98`
`Dense(200 => 200)/cpu/reverse/ReverseDiff (compiled)/(200, 128)`	`296119` ns	`297415` ns	`1.00`
`Dense(200 => 200)/cpu/reverse/Zygote/(200, 128)`	`268788.5` ns	`267444.5` ns	`1.01`
`Dense(200 => 200)/cpu/reverse/Tracker/(200, 128)`	`371350` ns	`369540` ns	`1.00`
`Dense(200 => 200)/cpu/reverse/ReverseDiff/(200, 128)`	`416614.5` ns	`411308` ns	`1.01`
`Dense(200 => 200)/cpu/reverse/Flux/(200, 128)`	`277620` ns	`277337.5` ns	`1.00`
`Dense(200 => 200)/cpu/reverse/SimpleChains/(200, 128)`	`413018` ns	`409664.5` ns	`1.01`
`Dense(200 => 200)/cpu/reverse/Enzyme/(200, 128)`	`83707` ns	`83486` ns	`1.00`
`Dense(200 => 200)/cpu/forward/NamedTuple/(200, 128)`	`83506` ns	`81302` ns	`1.03`
`Dense(200 => 200)/cpu/forward/ComponentArray/(200, 128)`	`85515.5` ns	`85018` ns	`1.01`
`Dense(200 => 200)/cpu/forward/Flux/(200, 128)`	`87824` ns	`87734` ns	`1.00`
`Dense(200 => 200)/cpu/forward/SimpleChains/(200, 128)`	`104536` ns	`104626` ns	`1.00`
`Conv((3, 3), 16 => 16)/cpu/reverse/ReverseDiff (compiled)/(64, 64, 16, 128)`	`210659613` ns	`208418135` ns	`1.01`
`Conv((3, 3), 16 => 16)/cpu/reverse/Zygote/(64, 64, 16, 128)`	`330079128.5` ns	`329863332.5` ns	`1.00`
`Conv((3, 3), 16 => 16)/cpu/reverse/Tracker/(64, 64, 16, 128)`	`431156129.5` ns	`437868758` ns	`0.98`
`Conv((3, 3), 16 => 16)/cpu/reverse/ReverseDiff/(64, 64, 16, 128)`	`483091023` ns	`473245652.5` ns	`1.02`
`Conv((3, 3), 16 => 16)/cpu/reverse/Flux/(64, 64, 16, 128)`	`406853666.5` ns	`409579507.5` ns	`0.99`
`Conv((3, 3), 16 => 16)/cpu/reverse/SimpleChains/(64, 64, 16, 128)`	`332576623.5` ns	`338434555` ns	`0.98`
`Conv((3, 3), 16 => 16)/cpu/reverse/Enzyme/(64, 64, 16, 128)`	`101453482.5` ns	`101758684` ns	`1.00`
`Conv((3, 3), 16 => 16)/cpu/forward/NamedTuple/(64, 64, 16, 128)`	`43970098` ns	`43942909` ns	`1.00`
`Conv((3, 3), 16 => 16)/cpu/forward/ComponentArray/(64, 64, 16, 128)`	`43875267` ns	`43793713` ns	`1.00`
`Conv((3, 3), 16 => 16)/cpu/forward/Flux/(64, 64, 16, 128)`	`64445085.5` ns	`57038485` ns	`1.13`
`Conv((3, 3), 16 => 16)/cpu/forward/SimpleChains/(64, 64, 16, 128)`	`28197406.5` ns	`28142581.5` ns	`1.00`
`Dense(2000 => 2000)/cpu/reverse/ReverseDiff (compiled)/(2000, 128)`	`19164597` ns	`19007086` ns	`1.01`
`Dense(2000 => 2000)/cpu/reverse/Zygote/(2000, 128)`	`19644039.5` ns	`19599865` ns	`1.00`
`Dense(2000 => 2000)/cpu/reverse/Tracker/(2000, 128)`	`23722580` ns	`23608296` ns	`1.00`
`Dense(2000 => 2000)/cpu/reverse/ReverseDiff/(2000, 128)`	`24303903` ns	`24199216.5` ns	`1.00`
`Dense(2000 => 2000)/cpu/reverse/Flux/(2000, 128)`	`19824011` ns	`19621295` ns	`1.01`
`Dense(2000 => 2000)/cpu/reverse/Enzyme/(2000, 128)`	`6536887` ns	`6523963` ns	`1.00`
`Dense(2000 => 2000)/cpu/forward/NamedTuple/(2000, 128)`	`6555702` ns	`6565571` ns	`1.00`
`Dense(2000 => 2000)/cpu/forward/ComponentArray/(2000, 128)`	`6563327` ns	`6584434` ns	`1.00`
`Dense(2000 => 2000)/cpu/forward/Flux/(2000, 128)`	`6509676` ns	`6525087.5` ns	`1.00`

This comment was automatically generated by workflow using github-action-benchmark.

avik-pal force-pushed the ap/fix_amdtests_takeii branch from 2bfe776 to dfbb550 Compare June 19, 2024 00:45

avik-pal force-pushed the ap/fix_amdtests_takeii branch 6 times, most recently from cc1def5 to fa32fdf Compare June 19, 2024 01:17

Try fix stalling for AMDGPU

0fbe669

avik-pal force-pushed the ap/fix_amdtests_takeii branch 3 times, most recently from 40c48a3 to 7138554 Compare June 19, 2024 01:38

Does loading LuxAMDGPU fix it?

756ca50

avik-pal force-pushed the ap/fix_amdtests_takeii branch 2 times, most recently from 26d8204 to ebeecd6 Compare June 19, 2024 01:53

Try fixing AMDGPU version

ad7acc0

avik-pal force-pushed the ap/fix_amdtests_takeii branch 2 times, most recently from a634036 to ad7acc0 Compare June 19, 2024 02:02

avik-pal linked an issue Jun 19, 2024 that may be closed by this pull request

AMDGPU CI stalls #715

Closed

github-actions bot reviewed Jun 19, 2024

View reviewed changes

avik-pal merged commit f1b8c12 into main Jun 19, 2024
52 of 53 checks passed

avik-pal deleted the ap/fix_amdtests_takeii branch June 19, 2024 02:53

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Try fixing AMDGPU test stalling #716

Try fixing AMDGPU test stalling #716

avik-pal commented Jun 19, 2024

codecov bot commented Jun 19, 2024 •

edited

Loading

github-actions bot left a comment

Try fixing AMDGPU test stalling #716

Try fixing AMDGPU test stalling #716

Conversation

avik-pal commented Jun 19, 2024

codecov bot commented Jun 19, 2024 • edited Loading

Codecov Report

github-actions bot left a comment

Choose a reason for hiding this comment

Benchmark Results

codecov bot commented Jun 19, 2024 •

edited

Loading