Text generation

[floatingbigcat]

support text generation with image as input.

• add image as a input of model for generate funciton
• add option for tokenizer to tokenize without pad and convert to tensor

Use instruction:

1. put the image_path and text prompt to jsonl file, set the config of text-generation.yml
   eg: sample_input.jsonl {'image':'image_path', 'text':'your_prompt'}
2. set up the environment just like the one for training.
3. run with python deepy.py generate.py -d your_train_configs(eg: 410M.yml summit-setup.yml) text-generation.yml

To do:
support the use_cache=True, which shall speed up the inference.

[daniel-z-kaplan]

Example on how to run:
python3 deepy.py generate.py -d configs magma_pythia_410M.yml magma_setup.yml text-generation.yml

Check text-generation for the input file name, make sure you have that file created, using the {image, text} dict setup described above.

You'll need to use your job.sh source, conda env if relevant, and export torch extensions also.