Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Use minimal softmax mask values based on the datatype (#966)
* added HF to NeoX 2.0 conversion script with mp and pp sharding * (1) added missing curly brace to pythial/1-4B config; (2) fixed a bug related to a hardcoded value withing the conversion script (3) fixed possible bugs in the conversion script wrt the mp sharding convention * fill in minimal possible mask values * initialize tensor on the target device --------- Co-authored-by: Quentin Anthony <[email protected]>
- Loading branch information