You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
DeepSpeed allows you to load HF models for inference easily using HF’s new pipeline API. We should verify that this works with NeoX as well, and add whatever QoL patches enable easy inference with HF models.
Note that NeoX currently only supports autoregressive models, while Megatron DeepSpeed supports encoder and encoder-decoder models as well. We will likely need to do some careful type checking (or add encoder support).
The text was updated successfully, but these errors were encountered:
DeepSpeed allows you to load HF models for inference easily using HF’s new
pipeline
API. We should verify that this works with NeoX as well, and add whatever QoL patches enable easy inference with HF models.Note that NeoX currently only supports autoregressive models, while Megatron DeepSpeed supports encoder and encoder-decoder models as well. We will likely need to do some careful type checking (or add encoder support).
The text was updated successfully, but these errors were encountered: