CUDA error: device-side assert triggered in self.forward_extend_multi_modal(batch)

[LetheRiver0]

• model: llava-v1.6-34b
• device: 2 H100

I have successfully started up backend for llava-v1.6-34b using similar code shown in https://github.com/sgl-project/sglang/blob/f6dbd24043b8c18d87a14b3c6fe5c4f567f6c1ba/examples/quick_start/srt_example_llava.py
Everything is ok when I run single or batch in one process. However, if I use parallel, like simultaneous run 2 batch function, There is a certain probability get error in backend, it doesn't appear every time, the full error message is shown below

I found some similar error in other issues, but it seems is different from my issue, dose anyone meet the same issue and know how to fix it? Thanks~

[LetheRiver0]

Does anyone know how to handle this?