Simplifications and some fun stuff for the MNIST Gluon tutorial

[kohr-h]

Description

This PR removes some of the (IMO) unnecessary complexity from the tutorial (one of the first that new users click on) by always using 1 GPU or 1 CPU. This makes the MLP training code much more straightforward (no splitting, no inner loop).
I also think that there was too much staccato of obvious comments in that part, while the truly non-obvious parts were left uncommented (e.g., what is ag.record()?). I tried to straighten up that part a bit, although I think some comments are still a bit on the obvious side.

Furthermore, I added a small section where some of the mislabeled (apparently it's one "l", need to fix that) images are plotted together with the predicted and true labels. I think that's more fun than just looking at an accuracy score.

I'm not finished with the whole tutorial yet, so it's WIP, but I'd like to know what you think before I put more work into this PR.

Note: I've included the 2 new images in the same directory as the tutorial for convenience, they would obviously need to go somewhere else.

Checklist

Essentials

Please feel free to remove inapplicable items for your PR.

• Changes are complete (i.e. I finished coding on this PR)

[ankkhedia]

@kohr-h Thanks for your contribiution!

@mxnet-label-bot [pr-work-in-progress, Gluon]

[ThomasDelteil]

Thanks for the much needed update @kohr-h, can I bother you to update the data loading to be pure Gluon rather than using the old DataIter API ? That means no need to reset the iterator between epoch and the data comes out of the iterator as a tuple (data, label) rather than a batch.data and batch.label

train_dataset = mx.gluon.data.vision.MNIST(train=True).transform_first(mx.gluon.data.vision.transforms.ToTensor())
test_dataset = mx.gluon.data.vision.MNIST(train=False).transform_first(mx.gluon.data.vision.transforms.ToTensor())

train_data = mx.gluon.data.DataLoader(train_dataset, shuffle=True, last_batch='rollover', batch_size=batch_size, num_workers=2)
test_data = mx.gluon.data.DataLoader(train_dataset, shuffle=False, last_batch='rollover', batch_size=batch_size, num_workers=2)

[kohr-h]

Thanks for the much needed update @kohr-h, can I bother you to update the data loading to be pure Gluon rather than using the old DataIter API ? That means no need to reset the iterator between epoch and the data comes out of the iterator as a tuple (data, label) rather than a batch.data and batch.label

No problem. I've started doing that, more to come.

[kalyc]

Thanks for your contribution @kohr-h
Could you please update the PR as per discussions above?

[kohr-h]

Thanks for the reminder @kalyc. I got too many balls in the air. Will get this done asap.

[kohr-h]

I'm done with the rewording and restructuring of the example.

Note that I also made the CNN part more basic by moving from a custom nn.Block subclass to a nn.Sequential model. I don't think it added any value, and there are other tutorials that explain this in more detail.

Ready for review.

[ThomasDelteil]

A lot of nit-picking on the formatting but overall when these are addressed I think we should be good to merge. Great work on the wording and explanations. Thanks for the contribution 👍

[kohr-h]

I think I addressed all the comments from the review.

[vandanavk]

@sandeep-krishnamurthy @stu1130 for review

[sandeep-krishnamurthy]

Looks great. Thank you.
1 question - Why activation function in last layer?

[kohr-h]

1 question - Why activation function in last layer?

No real reason. I think I didn't change the original logic, but I can check again. If a different last activation is more reasonable for this tutorial, I can put it in, no problem.

[sandeep-krishnamurthy]

Sigmoid activation is a better fit here to output the probability of 10 digits.

[kohr-h]

@sandeep-krishnamurthy I switched both nets to sigmoid activation. Note though that they perform worse with the same settings. I've tested a few learning rates, and the current ones improve things a bit. But maybe it's okay for the scope of the tutorial that the training isn't perfectly fine-tuned.

[kohr-h]

Okay, now we're in the right ballpark with the results. Thanks @ThomasDelteil for the catch.