D-PSGD

This repository implements Decentralized Parallel Stochastic Gradient Descent (D-PSGD) algorithm.

Available Models

--world_size = total number of agents
--graph = graph topology (default ring)
--neighbors = number of neighbor per agent (default 2)
--arch = model to train
--normtype = type of normalization layer
--dataset = dataset to train
--batch_size = batch size for training
--epochs = total number of training epochs
--lr = learning rate
--momentum = momentum coefficient
--nesterov = activates nesterov momentum
--gamma = averaging rate for gossip
--alpha = amount of skew in the data distribution (alpha of Dirichlet distribution); 0.01 = completely non-iid and 10 = more towards iid
--weight_decay = L2 regularization coefficient

test file shows sample commands to run the code

sh test.sh

Name		Name	Last commit message	Last commit date
Latest commit History 5 Commits
gossip		gossip
models		models
.gitignore		.gitignore
README.md		README.md
env.yml		env.yml
partition_data.py		partition_data.py
test.sh		test.sh
trainer.py		trainer.py