A surrogate model is an approximation method that mimics the behavior of a computationally expensive simulation. In more mathematical terms: suppose we are attempting to optimize a function f(p), but each calculation of f is very expensive. It may be the case we need to solve a PDE for each point or use advanced numerical linear algebra machinery, which is usually costly. The idea is then to develop a surrogate model g which approximates f by training on previous data collected from evaluations of f. The construction of a surrogate model can be seen as a three-step process:

1. Sample selection
2. Construction of the surrogate model
3. Surrogate optimization

• Grid
• Uniform
• Sobol
• Latin Hypercube
• Low Discrepancy
• Kronecker
• Golden
• Random

• Kriging
• Kriging using Stheno
• Radial Basis
• Wendland
• Linear
• Second Order Polynomial
• Support Vector Machines (Wait for LIBSVM resolution)
• Neural Networks
• Random Forests
• Lobachevsky
• Inverse-distance
• Polynomial expansions
• Variable fidelity
• Mixture of experts (Waiting GaussianMixtures package to work on v1.5)
• Earth
• Gradient Enhanced Kriging

• SRBF
• LCBS
• DYCORS
• EI
• SOP
• Multi-optimization: SMB and RTEA

using Pkg
Pkg.add("Surrogates")