All .onnx files from some folders models: 4xNomos8kSCSRFormer, 4xNomos8kSCHAT-S, 4xNomos8kSCHAT-L with all the backend

[kedaitinh12]

I can't use all .onnx files from some folders with all the backend:
https://github.com/Phhofm/models/tree/main/4xNomos8kSCSRFormer
https://github.com/Phhofm/models/tree/main/4xNomos8kSCHAT-S
https://github.com/Phhofm/models/tree/main/4xNomos8kSCHAT-L

Can you add support it to vs-mlrt? Thanks

[WolframRhodium]

The NCNN_VK backend does not support these models, while other backends should work (tested 4xNomos8kSCSRFormer.onnx).

[kedaitinh12]

Example with Cuda ver: only 4xNomos8kSCSRFormer_131616_onnxsim work and others (4xNomos8kSCSRFormer_136464_fp16_onnxsim, 4xNomos8kSCSRFormer_136464_fp16_onnxsim, 4xNomos8kSCSRFormer_131212_onnxsim_fp16, 4xNomos8kSCHAT-S, 4xNomos8kSCHAT-L) don't work for me

[WolframRhodium]

*_fp16_* indicates that the onnx files require fp16 IO. 4xNomos8kSCHAT-S and 4xNomos8kSCHAT-L also requires fp16 IO, although the author does not encode that in the filename.

At present only OV_* backends support specifying non-fp32 IO format in onnx. I may add support for this feature in ORT_* backends, but not in the TRT backend.

Please be specific about what models do not work and what's the error message in the future. Otherwise I will simply ignore the issue.

[hooke007]

In my previous test, I only remembered there are some problems with dynamic shape onnx files.
i.e. https://github.com/Phhofm/models/tree/main/2xHFA2kAVCOmniSR/onnx
It will report sth like 'negative value in shape'