I'm playing with PyTorch on the CIFAR10 dataset.
Because SyncBatchNorm only supports one process per GPU, so we need to launch #GPU process.
# e.g. 2 GPUs:
python -m torch.distributed.launch --nproc_per_node=2 main.py --world_size=2
- Python 3.6+
- PyTorch 1.0+
# Start training with:
python main.py
# You can manually resume the training with:
python main.py --resume --lr=0.01
Model | Acc. |
---|---|
VGG16 | 92.64% |
ResNet18 | 93.02% |
ResNet50 | 93.62% |
ResNet101 | 93.75% |
RegNetX_200MF | 94.24% |
RegNetY_400MF | 94.29% |
MobileNetV2 | 94.43% |
ResNeXt29(32x4d) | 94.73% |
ResNeXt29(2x64d) | 94.82% |
SimpleDLA | 94.89% |
DenseNet121 | 95.04% |
PreActResNet18 | 95.11% |
DPN92 | 95.16% |
DLA | 95.47% |