Skip to content

Latest commit

 

History

History
100 lines (67 loc) · 2.72 KB

README.md

File metadata and controls

100 lines (67 loc) · 2.72 KB

Paleo

Build Status License

Paleo is an analytical model to estimate the scalability and performance of deep learning systems. It can be used to

  • efficiently explore the space of scalable deep learning systems,
  • quickly diagnose their effectiveness for a given problem instance.

Live demo

Getting Started

Installation

Paleo uses the following dependencies:

  • numpy
  • click
  • six
  • cuDNN (Optional. Use --use_only_gemm to disable cuDNN heuristics)

Use pip to install the depenencies with the pinned versions:

pip install -r requirements.txt

Tested with Python 2.7, cuDNN v4 on Ubuntu 14.04.

To install Paleo, run the following command in the cloned directory:

python setup.py install

Usage

Paleo provides programmatic APIs to retrieve runtime estimations.

The following is an example of estimating SGD executions under strong scaling.

from paleo.profilers import BaseProfiler

class SGDProfiler(BaseProfiler):
    def __init__(self, filename):
        super(SGDProfiler, self).__init__(filename)

    def simulate(self, workers, batch_size=128):
        fwd_time, params_in_bytes = self.estimate_forward(batch_size //
                                                          workers)
        bwd_time = self.estimate_backward(batch_size // workers)
        update_time = self.estimate_update(params_in_bytes)

        t_comp = fwd_time + bwd_time + update_time
        t_comm = self.estimate_comm(workers, params_in_bytes)
        return t_comp + t_comm

Definitions

Model Architectures

Paleo uses a special json format to for model architecture specification. Predefined architectures can be found under the nets/ directory. Paleo also provides a convertor for Caffe prototxt format (see nets/caffe/ for details).

  • AlexNet v2
  • Inception v3
  • NiN
  • Overfeat
  • VGG-16
  • ResNet-50 (from Caffe spec)
  • DenseNet (from Caffe spec)

Hardware

Predefined hardware specificiations are in paleo/device.py.

Reference Paper

Hang Qi, Evan R. Sparks, and Ameet Talwalkar. Paleo: A Performance Model for Deep Neural Networks. International Conference on Learning Representations (ICLR), 2017.

@inproceedings{qi17paleo,
  author={Hang Qi and Evan R. Sparks and Ameet Talwalkar},
  booktitle={Proceedings of the International Conference on Learning Representations},
  title={Paleo: A Performance Model for Deep Neural Networks},
  year={2017}
}

License

Apache 2.0