A collection of gradient-based optimization methods.

In this repository, four gradient-based optimization methods are implemented, including:

• (Stochastic) Gradient Descent
• AdaGrad
• AdaDelta
• Adam

For the gradient computation, you can either rely on the whole dataset (where stochastic GD is transformed into GD), or only use a subset for faster computation and parameter update. This is controlled by the "batcSiz" option.

"demo.m" provides a demonstration of how to solve a linear regression problem via the above methods. Just run it!