Add MNIST example with CNN #485
Conversation
|
Right now I am using raw files for the model. I need to come up with a Python script which saves them to GGUF. Hence this is a draft. |
examples/mnist/main-cnn.cpp
Outdated
| // TODO: is there a better way to convert 1d bias to 4d bias? | ||
| for (int i = 0; i < ne; i++) { | ||
| for (int j = 0; j < ne0*ne1; j++) { | ||
| ggml_set_f32_1d(bias, i*ne0*ne1 + j, bias_f32[i]); | ||
| } | ||
| } |
There was a problem hiding this comment.
We probably need to add 4d overloads for ggml_set_... but for now this should be fine
Adding GGUF conversion could be a useful example, but it's also OK to keep reading |
Add one more implementation for MNIST which uses Conv2D layers, ref: https://keras.io/examples/vision/mnist_convnet/. It achieves ~99% accuracy on the MNIST test set and also performs better for user inputs. This implementation expects a model in GGUF format. You can get one with the 'mnist-cnn.py' script. Example usage: $ ./mnist-cnn.py train mnist-cnn-model ... Keras model saved to 'mnist-cnn-model' $ ./mnist-cnn.py convert mnist-cnn-model ... Model converted and saved to 'mnist-cnn-model.gguf' $ ./mnist-cnn mnist-cnn-model.gguf models/mnist/t10k-images.idx3-ubyte
8254e72
to
9a287f7
Compare
|
Two observations on this simple example:
|
|
I tested this and everything works correctly. Here is the graph plot:
Yes, it's always difficult to match the shapes during such conversions. I guess there are conventions in standard frameworks but
It's not related to LLMs. The convention for referencing the axes in the tensor is different between Python and # example 4d tensor in Python
print(x.shape)
(32, 1, 3, 3)
print(x.shape[0], kernel1.shape[1], kernel1.shape[2], kernel1.shape[3])
32 1 3 3// same tensor in ggml
printf("(%d, %d, %d, %d)\n"), x->ne[0], x->ne[1], x->ne[2], x->ne[3]);
(3, 3, 1, 32)So Shall we merge this or add a README.md first? |
|
I will update the README and the web app in a follow-up PR |
Add one more implementation for MNIST which uses Conv2D layers, ref: https://keras.io/examples/vision/mnist_convnet/
It achieves ~99% accuracy on the MNIST test set and also performs better for user inputs.