Official implementation of the paper "Progressive Transmission and Inference of Deep Learning Models", to appear at 20th IEEE International Conference on Machine Learning and Applications (ICMLA 2021).

[arXiv] [demo]

Youngsoo Lee¹, Sangdoo Yun², Yeonghun Kim¹ Sunghee Choi¹

¹ School of Computing, Korea Advanced Institute of Science and Technology(KAIST)
² AI LAB, NAVER Corp.

Modern image files are usually progressively transmitted and provide a preview before downloading the entire image for improved user experience to cope with a slow network connection. In this paper, with a similar goal, we propose a progressive transmission framework for deep learning models, especially to deal with the scenario where pre-trained deep learning models are transmitted from servers and executed at user devices (e.g., web browser or mobile). Our progressive transmission allows inferring approximate models in the middle of file delivery, and quickly provide an acceptable intermediate outputs. On the server-side, a deep learning model is divided and progressively transmitted to the user devices. Then, the divided pieces are progressively concatenated to construct approximate models on user devices. Experiments show that our method is computationally efficient without increasing the model size and total transmission time while preserving the model accuracy. We further demonstrate that our method can improve the user experience by providing the approximate models especially in a slow connection.

Image classification Model (MobileNetV2):

Object Detection Model (SSD-MobileNetV2):

Demo: You can try using our pre-built models in https://prev.github.io/progressivenet

ProgressiveNet is implemented based on TensorFlowJS, which is an open-source hardware-accelerated JavaScript library for training and deploying machine learning models. Our implementation interacts with the TFJS core engine, by passing the intermediate weight matrices to the TFJS and receives the approximiate results in the middle of the transmission.

$ npm install progressivenet

import { ProgressiveNet, Classifier } from 'progressivenet';

const pgNet = new ProgressiveNet({
    modelUrl: '<model_url>',
    numProgressSteps: 8,
});
const img = document.getElementById('my-img');

pgNet.loadSequentially((model, isLast, progressStep) => {
    const classifier = new Classifier(model);
    const predictions = await classifier.classify(img);
    console.log(`Result from step #${progressStep}: ${predictions}`);

}).then() => {
    console.log('All models are transmitted and inferenced');
}

import { ProgressiveNet, ObjectDetector } from 'progressivenet';

const pgNet = new ProgressiveNet({ modelUrl: '<model_url>' });
const img = document.getElementById('my-img');

pgNet.loadSequentially((model, isLast, progressStep) => {
    const detector = new ObjectDetector(model);
    const predictions = await detector.detect(img);
    console.log(`Result from step #${progressStep}: ${predictions}`);

}).then() => {
    console.log('All models are transmitted and inferenced');
}

You can see the detailed code in demo. It is written in TypeScript and implemented with jQuery.