I am a developer learning fast.ai with no good math together with other coders @CoderBunker, Shanghai

vgg = Vgg16() 

batches = vgg.get_batches(path+'train', batch_size=batch_size) 
val_batches = vgg.get_batches(path+'valid', batch_size=batch_size*2)

vgg.finetune(batches) 
vgg.fit(batches, val_batches, nb_epoch=1)

The running result: loss: 0.1229 - acc: 0.9672 - val_loss: 0.0581 - val_acc: 0.9820

To solve such complex problem (image classification), you may feel above codes too simple just like me. This is the pain of top-down learning approach, what behind "whole-picture" is a black magic box. As we ask what's behind this black-box, we are asking 'top-down' questions in problem-solving way. Then, tackle these questions in a 'bottom-up' way would make our learning practical and effective.

• How computer learns the weights and bias (W, b) of each pixel on an image dataset from training data? solved
• And how the classification is calculated on test data for predication? solved
• Why softmax regression for image classification? []

Jeremy uses excel sheet to simulate deep learning on linear regression (y=Wx+b). The W and b are simple real value (2,30), y=2x + 30. https://www.youtube.com/watch?v=qnoLMkosHuE&pbjreload=10&app=desktop

https://classroom.udacity.com/courses/ud120/lessons/2301748537/concepts/24575785420923

https://www.tensorflow.org/get_started/get_started#tensorflow_core_tutorial

• Simple code with simple data. Learn by crafting the codes with very simple data set;
• Learning with reports, we can visualize the learning process with tensorboard. As a result of learning y=Wx+b, W: [-0.9999969] b: [ 0.99999082] loss: 5.69997e-11

https://www.tensorflow.org/get_started/mnist/beginners MNIST problem solved with softmax regression. The explanation pictures are very helpful for beginners. For example, below is softmax regression scalargraph,

Training Labels (mnist.train.labels) is [55000, 10] matrix of floats ;

Weights is [784, 10] matrix of floats (given a certain pixel and known number, what's the weight); while bias is [10] array of floats;

The training process is a process of calculating unknown Weights and bias by given known lables on images; pseudo equation as below,

// train.y is known labels on images, a [55000, 10] matrix of float;
// train.x is known image pixel intensities, a [55000, 784] matrix of float;
// W is unknown weights, a [784, 10] matrix of float;
// b is unknown bias, a [10] array of float;

train.y = train.x * W (?) + b (?);

// after training, we will get W and b;

Here with visualization of trained weights by numbers [0,1,..,9],

The classification is a process of calculating unknown label by given trained weights and bias upon one image. pseudo equation as below,

// test.y is unknown label, a [10] array of float;
// test.x is known image pixel intensities, a [1, 784] matrix of float, or simply a [784] array of float;
// W is trained weights, a [784,10] matrix of float;
// b is trained bias, a [10] array of float;

test.y (?) = test.x * W + b 

// after calculating, we will get a [10] array of probabilities on classification (and most likely which number in [0..9])

The kaggle competition assignment, https://wiki.fast.ai/index.php/Lesson_1#Overview_of_homework_assignment

Stuck at 12. write a script that takes the predict() data of the new Dogs vs. Cats data and writes the data to a new csv file in the format of the sample_submission.csv file that was downloaded with the Dogs vs. Cats

The source code w. explanation is not explicitly mentioned in lesson1, it's here, https://github.com/fastai/courses/blob/master/deeplearning1/nbs/dogs_cats_redux.ipynb

• How can I know whether jupyter kernel is running while sometimes it doesnot response (any command available)? [solved] Use 'htop' command, check if the jupyter process is still running
• How to have jeremy's theme on jupyter notebook (fold/unfold on sections)?
• The test folder contains 12,498 images, not 12500 images? [solved] most likely miss files while download, download again and check with '$ls | wc -l' after download

Digg into VGG_16.py, questions come out

def VGG_16();

...

model.add(Dense(1000, activation='softmax'))

...

https://www.thoughtly.co/blog/deep-learning-lesson-2/

softmax is a generalization of logistic function that “squashes”(maps) a K-dimensional vector z of arbitrary real values to a K-dimensional vector σ(z) of real values in the range (0, 1) that add up to 1. e.g. [2.1, 3.4, 81, 13.5] => [0.021, 0.034, 0.81, 0.135]

https://www.quora.com/Artificial-Neural-Networks-Why-do-we-use-softmax-function-for-output-layer

https://www.quora.com/What-is-the-role-of-rectified-linear-ReLU-activation-function-in-CNN

https://colah.github.io/posts/2015-08-Backprop/

https://stackoverflow.com/questions/35613249/using-ipdb-to-debug-python-code-in-one-cell-jupyter-or-ipython

Questions:

• Where is Optim_Tutorial.ipynb shown in video lesson?
• What does model.DropOut mean?
• What if I rotate my handwrite number 3 for a MNIST CNN to recognize? The CNN predicts wrong number as 7.

A Quick Introduction to Neural Networks, https://ujjwalkarn.me/2016/08/09/quick-intro-neural-networks/ An Intuitive Explanation of Convolutional Neural Networks, https://ujjwalkarn.me/2016/08/11/intuitive-explanation-convnets/

https://www.quora.com/What-is-the-next-step-beyond-deep-learning-in-AI generalization