How to convert a model parallel model to hugging face model?

[guozhiyao]

Is your feature request related to a problem? Please describe.
I train a model with "model-parallel-size": 2, and try to convert it to hugging-face model. I refer to tools/convert_to_hf.py for conversion, and the model can load parameters normally, but the generated results are random.
While I train a model with "model-parallel-size": 1, and use the same code to convert and generate, the result is normal. So I suspect it is because of the conversion code.

Describe the solution you'd like
A clear and concise description of what you want to happen.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

[Quentin-Anthony]

Closing as duplicate of #880