How to use Beam Search for text generation?

[cdj0311]

Hi,
Thanks for this code. But I want know how to use Beam Search for text generation?

[StellaAthena]

We do not currently support beam search for text generation. You will have to implement that yourself or use the WIP HuggingFace port

[cdj0311]

We do not currently support beam search for text generation. You will have to implement that yourself or use the WIP HuggingFace port

Thanks, but have a script convert model to huggingface transformers load?

[StellaAthena]

The model is on the EleutherAI HF page already.

[cdj0311]

The model is on the EleutherAI HF page already.

enn, the 20B model on the HF page is too large to loaded, I trained a small model (with small.yml), how to convert it ?

[StellaAthena]

@zphang where’s the function you wrote for this? I don’t see it in tools like I expected to.

[jason9693]

@cdj0311 you can use this script.
I've tested on 1B and 3B models and both of all is succeed.

#668