Each BNN object must be instantiated with a config.json file, which is a dictionary of hyperparameters and other relevant variables. Here is a complete list of all keys and their associated set of values. (Our paper, as well as papers from which we implemented the inference algorithms, are all linked in the README.)

General variables

• "prior_type": String. Which prior formulation to use. Use "oc_negative"/"oc_positive" for negative/positive constraints respectively, or "gaussian" for the vanilla isotropic Gaussian prior.
• "use_tensorboard": Boolean. true for TensorBoardX support. Tensorboard runs on your local machine and is useful for real-time monitoring of variables like ELBO and log-likelihood during the training process.

Neural network variables

• "architecture": Integer List. Defines hidden layers of the BNN. E.g. [10,10] will create a BNN with two hidden layers of 10 nodes each. Excludes input and output layers.
• "sigma_noise": Float. Standard deviation of environmental noise, modeled as Gaussian. Applicable for regression tasks only.
• "sigma_w": Float. Gaussian standard deviation of network parameters, used for both initialization of parameters and for computing the isotropic Gaussian prior term.
• "activation": String. Type of non-linearity applied to hidden layers. Currently only "rbf" supported, but it is easier to write your own function.
• "nbatches": Integer. How many batches to divide the training set into, for batched training. Use 0 for unbatched training.

Hyperparameters for positive constraint prior

• "ocp_nsamples": Integer. Number of input points to sample to compute constraint prior term.
• "ocp_bimodal": Boolean. true to use a bimodal Gaussian as the constraint prior (instead of unimodal Gaussian). Applicable for regression cases only. Will work only with SVGD, which is the only inference method that can capture multimodal posteriors.
• "ocp_sigma_y": Float. Standard deviation for a Gaussian constraint prior. Applicable for regression cases only.
• "ocp_alpha": Float. Hyperparameter for Dirichlet constraint prior. See paper for more details. Applicable for classification task only.
• "ocp_gamma": Integer. Multiplier for Dirichlet constraint prior. Applicable for classification task only.

Hyperparameters for negative constraint prior (regression only)

• "ocn_nsamples": Integer. Number of input points to sample to compute constraint prior term.
• "ocn_gamma": Integer. Hyperparameter for negative constraint prior. See paper for more details.
• "ocn_tau": Int/Float List. Hyperparameter for negative constraint prior. See paper for more details. List should contain 2 values only.

Hyperparameters for SVGD (see SVGD paper for more details)

• "svgd_nparticles": Integer. Number of particles to use (equivalent to number of posterior samples collected).
• "svgd_epochs": Integer. Number of training epochs to run.
• "svgd_init_lr": Float. Initial learning rate for Adagrad optimizer.

Hyperparameters for HMC (see HMC paper for more details)

• "hmc_nburnin": Integer. Number of burn-in iterations.
• "hmc_nsamples": Integer. Number of actual samples to collect after burn-in.
• "hmc_ninterval": Integer. Sampling period. Hence the total number of iterations run altogether is:
  (hmc_nburnin + hmc_nsamples * hmc_ninterval)
• "hmc_epsilon": Float. Stepsize hyperparameter for HMC.
• "hmc_l": Integer. Trajectory length hyperparameter for HMC.

Hyperparameters for BBB (see BBB paper for more details)

• "bbb_epochs": Integer. Number of training epochs to run.
• "bbb_init_mean": Float. Mean of Gaussian variational distribution, to initialize the first set of network parameters.
• "bbb_init_std": Float. Log-standard deviation of Gaussian variational distribution, to initialize the first set of network parameters. As per the BBB paper, this is in log space.
• "bbb_init_lr": Float. Initial learning rate for Adagrad optimizer.
• "bbb_esamples": Integer. Number of variable samples to draw each training epoch to approximate the ELBO.
• "bbb_nsamples": Integer. Number of posterior samples to draw from the optimized variational distribution. Used for post-inference plotting/evaluation only.

Hyperparameters for SGLD (see SGLD paper for more details)

• "sgld_nburnin": Integer. Number of burn-in iterations.
• "sgld_nsamples": Integer. Number of actual samples to collect after burn-in.
• "sgld_ninterval": Integer. Sampling period. Hence the total number of iterations run altogether is:
  (sgld_nburnin + sgld_nsamples * sgld_ninterval)
• "sgld_epa": Float. Hyperparameter to compute the stepsize each iteration.
• "sgld_epb": Float. Hyperparameter to compute the stepsize each iteration.
• "sgld_epgamma": Float. Hyperparameter to compute the stepsize each iteration.

Plotting variables

• "plot_fontsize": Integer. Matplotlib font size for title and axes labels.
• "plot_figsize": Int/Float List. Matplotlib figure size. List should contain 2 values only (width, height).
• "plot_dpi": Integer. Resolution to save the plot in.