This hands-on programming tutorial with step by step implementation of Bloom Large Language Text Generation Model will guide you building your own text generation application in Google Colab environment.
You will also learn to solve 3 different problems while working on this tutorial:
- RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0! (when checking argument for argument index in method wrapper__index_select)
Solution:
- The models were instantiated on CPU however the tokenization was done at the GPU so when the language modeling request was made the tensors were not found at one place either in CPU or on GPU. The solution was to restart the environment and reload all the resources properperly at the GPU.
- AttributeError: 'BaseModelOutputWithPastAndCrossAttentions' object has no attribute 'logits'
Solution:
- The language model object which you have loaded may or may not have the language modeling function so this error may occur if you do now use the correct model with language modeling support. In our case the first model we loaded does not have the correct language modelling support so we changed the class to load the model with language modeling support and it worked.
- GPU Memory Exception while loading Bloom models with tokenizer
Solution:
- Because I am using free version of Google Colab which has around 1GB GPU RAM so loading multiple large models cause memory exception. The solution is to load only those models which can be fit into the given GPU memory or use the CPU if that is an acceptable solution.
| 🔥 Bloom (Text Generation Large Language Model - LLM): Step by step implementation |
|
|
Latest Bloom LLM Model (176B Paramters):