Adapted from the original Python/PyTorch implementation at https://sirupsen.com/napkin/neural-net.

May 2022 / Markus Konrad [email protected]

The main file neuralnets.jl shows an example implementation of a single layer neural network with Julia by following a tutorial on "napkin math". It uses the AutoGrad package for calculating the loss function gradients for stochastic gradient descend (SGD). For a better understanding of AutoGrad two more files are added: autograd_tanh.jl shows how to get the derivate of a user defined function and autograd_lm.jl shows how to apply SGD for a simple linear model with two parameters.

You should run the code line-by-line with a Julia REPL in order to understand it.

The source-code is provided under Apache License 2.0 (see LICENSE file).