Add activation functions doc reference (Rebase #694) #702

avik-pal · 2024-06-15T20:26:58Z

No description provided.

github-actions

Benchmark Results

Benchmark suite	Current: `048c987`	Previous: `60ff714`	Ratio
`Dense(2 => 2)/cpu/reverse/ReverseDiff (compiled)/(2, 128)`	`4428.125` ns	`3713.125` ns	`1.19`
`Dense(2 => 2)/cpu/reverse/Zygote/(2, 128)`	`7142` ns	`7203.333333333333` ns	`0.99`
`Dense(2 => 2)/cpu/reverse/Tracker/(2, 128)`	`20834.5` ns	`20819` ns	`1.00`
`Dense(2 => 2)/cpu/reverse/ReverseDiff/(2, 128)`	`9864.8` ns	`9780.75` ns	`1.01`
`Dense(2 => 2)/cpu/reverse/Flux/(2, 128)`	`9104.75` ns	`9093` ns	`1.00`
`Dense(2 => 2)/cpu/reverse/SimpleChains/(2, 128)`	`4553.625` ns	`4467.125` ns	`1.02`
`Dense(2 => 2)/cpu/reverse/Enzyme/(2, 128)`	`1169.3956834532373` ns	`1159.7676056338028` ns	`1.01`
`Dense(2 => 2)/cpu/forward/NamedTuple/(2, 128)`	`1110.6` ns	`1169.8357664233577` ns	`0.95`
`Dense(2 => 2)/cpu/forward/ComponentArray/(2, 128)`	`1199.1153846153845` ns	`1189.0078125` ns	`1.01`
`Dense(2 => 2)/cpu/forward/Flux/(2, 128)`	`1816.7115384615386` ns	`1776.9912280701756` ns	`1.02`
`Dense(2 => 2)/cpu/forward/SimpleChains/(2, 128)`	`180.14425770308122` ns	`180.60507757404795` ns	`1.00`
`Dense(20 => 20)/cpu/reverse/ReverseDiff (compiled)/(20, 128)`	`17342` ns	`17362` ns	`1.00`
`Dense(20 => 20)/cpu/reverse/Zygote/(20, 128)`	`16932` ns	`16872` ns	`1.00`
`Dense(20 => 20)/cpu/reverse/Tracker/(20, 128)`	`39454` ns	`39698.5` ns	`0.99`
`Dense(20 => 20)/cpu/reverse/ReverseDiff/(20, 128)`	`29206` ns	`29275` ns	`1.00`
`Dense(20 => 20)/cpu/reverse/Flux/(20, 128)`	`20148` ns	`20047` ns	`1.01`
`Dense(20 => 20)/cpu/reverse/SimpleChains/(20, 128)`	`17423` ns	`17413` ns	`1.00`
`Dense(20 => 20)/cpu/reverse/Enzyme/(20, 128)`	`4358.142857142857` ns	`4328.785714285714` ns	`1.01`
`Dense(20 => 20)/cpu/forward/NamedTuple/(20, 128)`	`3876.125` ns	`3864.75` ns	`1.00`
`Dense(20 => 20)/cpu/forward/ComponentArray/(20, 128)`	`3952.5` ns	`3916.125` ns	`1.01`
`Dense(20 => 20)/cpu/forward/Flux/(20, 128)`	`4868.5` ns	`4892` ns	`1.00`
`Dense(20 => 20)/cpu/forward/SimpleChains/(20, 128)`	`1657.1` ns	`1654.1` ns	`1.00`
`Conv((3, 3), 3 => 3)/cpu/reverse/ReverseDiff (compiled)/(64, 64, 3, 128)`	`38631820` ns	`39421463` ns	`0.98`
`Conv((3, 3), 3 => 3)/cpu/reverse/Zygote/(64, 64, 3, 128)`	`57683377` ns	`57774528` ns	`1.00`
`Conv((3, 3), 3 => 3)/cpu/reverse/Tracker/(64, 64, 3, 128)`	`75969285.5` ns	`72442185` ns	`1.05`
`Conv((3, 3), 3 => 3)/cpu/reverse/ReverseDiff/(64, 64, 3, 128)`	`88583411` ns	`89245855` ns	`0.99`
`Conv((3, 3), 3 => 3)/cpu/reverse/Flux/(64, 64, 3, 128)`	`72392434` ns	`73071392` ns	`0.99`
`Conv((3, 3), 3 => 3)/cpu/reverse/SimpleChains/(64, 64, 3, 128)`	`12040945.5` ns	`12092880` ns	`1.00`
`Conv((3, 3), 3 => 3)/cpu/reverse/Enzyme/(64, 64, 3, 128)`	`18021764` ns	`17868830.5` ns	`1.01`
`Conv((3, 3), 3 => 3)/cpu/forward/NamedTuple/(64, 64, 3, 128)`	`7021798` ns	`7042694` ns	`1.00`
`Conv((3, 3), 3 => 3)/cpu/forward/ComponentArray/(64, 64, 3, 128)`	`6993652` ns	`7026994` ns	`1.00`
`Conv((3, 3), 3 => 3)/cpu/forward/Flux/(64, 64, 3, 128)`	`9921788` ns	`10167053.5` ns	`0.98`
`Conv((3, 3), 3 => 3)/cpu/forward/SimpleChains/(64, 64, 3, 128)`	`6407893` ns	`6398143` ns	`1.00`
`vgg16/cpu/reverse/Zygote/(32, 32, 3, 16)`	`719273349` ns	`733077817` ns	`0.98`
`vgg16/cpu/reverse/Zygote/(32, 32, 3, 64)`	`2544006669` ns	`2576408758` ns	`0.99`
`vgg16/cpu/reverse/Zygote/(32, 32, 3, 2)`	`127815071` ns	`145382696` ns	`0.88`
`vgg16/cpu/reverse/Tracker/(32, 32, 3, 16)`	`799066475` ns	`793204426` ns	`1.01`
`vgg16/cpu/reverse/Tracker/(32, 32, 3, 64)`	`2917659912` ns	`2934972747` ns	`0.99`
`vgg16/cpu/reverse/Tracker/(32, 32, 3, 2)`	`213494102` ns	`200283848.5` ns	`1.07`
`vgg16/cpu/reverse/Flux/(32, 32, 3, 16)`	`642963873.5` ns	`657077422.5` ns	`0.98`
`vgg16/cpu/reverse/Flux/(32, 32, 3, 64)`	`2601768257` ns	`2630726384.5` ns	`0.99`
`vgg16/cpu/reverse/Flux/(32, 32, 3, 2)`	`121558309` ns	`125267603` ns	`0.97`
`vgg16/cpu/forward/NamedTuple/(32, 32, 3, 16)`	`174060544` ns	`174556269.5` ns	`1.00`
`vgg16/cpu/forward/NamedTuple/(32, 32, 3, 64)`	`647696801` ns	`655564353.5` ns	`0.99`
`vgg16/cpu/forward/NamedTuple/(32, 32, 3, 2)`	`45915604.5` ns	`34840242` ns	`1.32`
`vgg16/cpu/forward/ComponentArray/(32, 32, 3, 16)`	`165079455.5` ns	`165209849.5` ns	`1.00`
`vgg16/cpu/forward/ComponentArray/(32, 32, 3, 64)`	`656049315` ns	`639944606` ns	`1.03`
`vgg16/cpu/forward/ComponentArray/(32, 32, 3, 2)`	`30219713.5` ns	`30130274.5` ns	`1.00`
`vgg16/cpu/forward/Flux/(32, 32, 3, 16)`	`186411468` ns	`186248429.5` ns	`1.00`
`vgg16/cpu/forward/Flux/(32, 32, 3, 64)`	`716921688.5` ns	`716769749` ns	`1.00`
`vgg16/cpu/forward/Flux/(32, 32, 3, 2)`	`35530661` ns	`35937506` ns	`0.99`
`Conv((3, 3), 64 => 64)/cpu/reverse/ReverseDiff (compiled)/(64, 64, 64, 128)`	`1227186714.5` ns	`1212793695` ns	`1.01`
`Conv((3, 3), 64 => 64)/cpu/reverse/Zygote/(64, 64, 64, 128)`	`1869529731` ns	`1876991316.5` ns	`1.00`
`Conv((3, 3), 64 => 64)/cpu/reverse/Tracker/(64, 64, 64, 128)`	`2394676167` ns	`2315042302` ns	`1.03`
`Conv((3, 3), 64 => 64)/cpu/reverse/ReverseDiff/(64, 64, 64, 128)`	`2539507689` ns	`2546229569` ns	`1.00`
`Conv((3, 3), 64 => 64)/cpu/reverse/Flux/(64, 64, 64, 128)`	`1829409974` ns	`1829086938.5` ns	`1.00`
`Conv((3, 3), 64 => 64)/cpu/reverse/Enzyme/(64, 64, 64, 128)`	`561098893` ns	`562582630` ns	`1.00`
`Conv((3, 3), 64 => 64)/cpu/forward/NamedTuple/(64, 64, 64, 128)`	`324618885` ns	`322654245` ns	`1.01`
`Conv((3, 3), 64 => 64)/cpu/forward/ComponentArray/(64, 64, 64, 128)`	`322829183` ns	`324527121` ns	`0.99`
`Conv((3, 3), 64 => 64)/cpu/forward/Flux/(64, 64, 64, 128)`	`456119879` ns	`368705304` ns	`1.24`
`Conv((3, 3), 1 => 1)/cpu/reverse/ReverseDiff (compiled)/(64, 64, 1, 128)`	`11844904` ns	`12030571` ns	`0.98`
`Conv((3, 3), 1 => 1)/cpu/reverse/Zygote/(64, 64, 1, 128)`	`17775098.5` ns	`17884778.5` ns	`0.99`
`Conv((3, 3), 1 => 1)/cpu/reverse/Tracker/(64, 64, 1, 128)`	`19178988` ns	`19210928.5` ns	`1.00`
`Conv((3, 3), 1 => 1)/cpu/reverse/ReverseDiff/(64, 64, 1, 128)`	`23866015` ns	`23885393.5` ns	`1.00`
`Conv((3, 3), 1 => 1)/cpu/reverse/Flux/(64, 64, 1, 128)`	`17827131` ns	`17872013` ns	`1.00`
`Conv((3, 3), 1 => 1)/cpu/reverse/SimpleChains/(64, 64, 1, 128)`	`1170115` ns	`1165770.5` ns	`1.00`
`Conv((3, 3), 1 => 1)/cpu/reverse/Enzyme/(64, 64, 1, 128)`	`5893353` ns	`5886802` ns	`1.00`
`Conv((3, 3), 1 => 1)/cpu/forward/NamedTuple/(64, 64, 1, 128)`	`2047065.5` ns	`2059749` ns	`0.99`
`Conv((3, 3), 1 => 1)/cpu/forward/ComponentArray/(64, 64, 1, 128)`	`2029506` ns	`2042150` ns	`0.99`
`Conv((3, 3), 1 => 1)/cpu/forward/Flux/(64, 64, 1, 128)`	`2069167` ns	`2090230` ns	`0.99`
`Conv((3, 3), 1 => 1)/cpu/forward/SimpleChains/(64, 64, 1, 128)`	`200476` ns	`205073` ns	`0.98`
`Dense(200 => 200)/cpu/reverse/ReverseDiff (compiled)/(200, 128)`	`292979` ns	`295001` ns	`0.99`
`Dense(200 => 200)/cpu/reverse/Zygote/(200, 128)`	`265457` ns	`266698` ns	`1.00`
`Dense(200 => 200)/cpu/reverse/Tracker/(200, 128)`	`367349` ns	`370442` ns	`0.99`
`Dense(200 => 200)/cpu/reverse/ReverseDiff/(200, 128)`	`408496` ns	`411668` ns	`0.99`
`Dense(200 => 200)/cpu/reverse/Flux/(200, 128)`	`274225` ns	`276035.5` ns	`0.99`
`Dense(200 => 200)/cpu/reverse/SimpleChains/(200, 128)`	`410580` ns	`410075` ns	`1.00`
`Dense(200 => 200)/cpu/reverse/Enzyme/(200, 128)`	`83146` ns	`83606` ns	`0.99`
`Dense(200 => 200)/cpu/forward/NamedTuple/(200, 128)`	`81332.5` ns	`81742` ns	`0.99`
`Dense(200 => 200)/cpu/forward/ComponentArray/(200, 128)`	`81673` ns	`82474` ns	`0.99`
`Dense(200 => 200)/cpu/forward/Flux/(200, 128)`	`87003` ns	`87192` ns	`1.00`
`Dense(200 => 200)/cpu/forward/SimpleChains/(200, 128)`	`104737` ns	`104675` ns	`1.00`
`Conv((3, 3), 16 => 16)/cpu/reverse/ReverseDiff (compiled)/(64, 64, 16, 128)`	`188649966.5` ns	`189862618.5` ns	`0.99`
`Conv((3, 3), 16 => 16)/cpu/reverse/Zygote/(64, 64, 16, 128)`	`321423770.5` ns	`323925648.5` ns	`0.99`
`Conv((3, 3), 16 => 16)/cpu/reverse/Tracker/(64, 64, 16, 128)`	`361037664` ns	`396072434` ns	`0.91`
`Conv((3, 3), 16 => 16)/cpu/reverse/ReverseDiff/(64, 64, 16, 128)`	`457294169` ns	`457714601.5` ns	`1.00`
`Conv((3, 3), 16 => 16)/cpu/reverse/Flux/(64, 64, 16, 128)`	`386837502` ns	`374592898` ns	`1.03`
`Conv((3, 3), 16 => 16)/cpu/reverse/SimpleChains/(64, 64, 16, 128)`	`328906990` ns	`346883743` ns	`0.95`
`Conv((3, 3), 16 => 16)/cpu/reverse/Enzyme/(64, 64, 16, 128)`	`102304223.5` ns	`101249018.5` ns	`1.01`
`Conv((3, 3), 16 => 16)/cpu/forward/NamedTuple/(64, 64, 16, 128)`	`43939177` ns	`43995641` ns	`1.00`
`Conv((3, 3), 16 => 16)/cpu/forward/ComponentArray/(64, 64, 16, 128)`	`43665266` ns	`43836879` ns	`1.00`
`Conv((3, 3), 16 => 16)/cpu/forward/Flux/(64, 64, 16, 128)`	`59671269` ns	`60056689` ns	`0.99`
`Conv((3, 3), 16 => 16)/cpu/forward/SimpleChains/(64, 64, 16, 128)`	`28303606` ns	`28759179` ns	`0.98`
`Dense(2000 => 2000)/cpu/reverse/ReverseDiff (compiled)/(2000, 128)`	`18826163` ns	`19178597` ns	`0.98`
`Dense(2000 => 2000)/cpu/reverse/Zygote/(2000, 128)`	`19568039` ns	`19643348.5` ns	`1.00`
`Dense(2000 => 2000)/cpu/reverse/Tracker/(2000, 128)`	`23494430` ns	`23527065` ns	`1.00`
`Dense(2000 => 2000)/cpu/reverse/ReverseDiff/(2000, 128)`	`24177864` ns	`24189371` ns	`1.00`
`Dense(2000 => 2000)/cpu/reverse/Flux/(2000, 128)`	`19692200.5` ns	`19709627` ns	`1.00`
`Dense(2000 => 2000)/cpu/reverse/Enzyme/(2000, 128)`	`6549963` ns	`6556235` ns	`1.00`
`Dense(2000 => 2000)/cpu/forward/NamedTuple/(2000, 128)`	`6551607.5` ns	`6556525` ns	`1.00`
`Dense(2000 => 2000)/cpu/forward/ComponentArray/(2000, 128)`	`6519504.5` ns	`6516631` ns	`1.00`
`Dense(2000 => 2000)/cpu/forward/Flux/(2000, 128)`	`6523951` ns	`6542791.5` ns	`1.00`

This comment was automatically generated by workflow using github-action-benchmark.

NeroBlackstone and others added 10 commits June 15, 2024 13:24

add activation functions doc reference

bcad37a

move activation function doc to Building blocks

fce835d

update docs for Parallel

fdad391

change Parallel docs

7e4c118

add FlattenLayer Example

f299ba0

add NoOpLayer example

536a549

add ReshapeLayer docs

6d08a31

update docs

6a98290

Update src/layers/containers.jl

a1b1fde

Run formatter

048c987

avik-pal mentioned this pull request Jun 15, 2024

Add activation functions doc reference #694

Closed

github-actions bot reviewed Jun 15, 2024

View reviewed changes

avik-pal merged commit 163098a into main Jun 15, 2024
67 of 71 checks passed

avik-pal deleted the rebase_docs branch June 15, 2024 21:30

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add activation functions doc reference (Rebase #694) #702

Add activation functions doc reference (Rebase #694) #702

avik-pal commented Jun 15, 2024

github-actions bot left a comment

Add activation functions doc reference (Rebase #694) #702

Add activation functions doc reference (Rebase #694) #702

Conversation

avik-pal commented Jun 15, 2024

github-actions bot left a comment

Choose a reason for hiding this comment

Benchmark Results