An implementation of using Singular Value Decomposition to accelerate training of neural networks.

Implemented in Julia using the Flux ML Library.

See the Report for technical details and comparisons (Skip to section 3.3 for the relevant parts)

See Examples.

Tested on Julia 1.3.0 and 1.4.1 with the following required packages.

• Flux: v0.10.4
• Zygote: v0.4.20
• TimerOutputs: v0.5.5 (TimerOutputs is required to quantify the difference in speeds between a normal DNN and SVD-DNN, it can be easily modified to not require TimerOutputs)

IJulia is required for the jupyter notebook examples.

Some of the examples may require the following packages (version numbers indicate the ones tested on, it may work on other versions too).

• HDF5: v0.13.2
• Images: v0.22.2
• MLDatasets: v0.5.2
• Plots: v1.2.4

Thanks to Abhijit Mohanty and Srinidhi Temkar for helping in the project.