12 Pull requests merged by 8 people

• fix python version and pytest install

  #1234 merged Jun 19, 2024

• Conversion script bugfixes

  #1218 merged Jun 7, 2024

• Fix changed behavior of pipe_parallel

  #1219 merged Jun 7, 2024

• fix conversion of hf -> neox for pythia in model parallel

  #1220 merged Jun 7, 2024

• Change python invocation syntax

  #1223 merged Jun 5, 2024

• init changes to README

  #1232 merged Jun 5, 2024

• add workflow_dispatch to gh actions pr so we can run on command

  #1233 merged Jun 4, 2024

• Fix markdown formatting error

  #1217 merged May 26, 2024

• Small tidying

  #1222 merged May 21, 2024

• fixed fused_rope naming in JIT + Readme

  #1224 merged May 21, 2024

• Add Torch Profiler Support

  #1226 merged May 21, 2024

• Rwkv pipeline parallelism

  #1221 merged May 21, 2024

3 Pull requests opened by 3 people

• Add lora support

  #1225 opened May 20, 2024

• Add Transformer Engine's version of RMSNorm and LayerNorm

  #1235 opened Jun 11, 2024

• Add tensor parallelism for RWKV

  #1237 opened Jun 19, 2024

2 Issues closed by 2 people

• Add Basic RWKV Block to GPT-NeoX

  #1167 closed Jun 19, 2024

• Cannot perform inference, be it unconditional. input-file or interactive

  #1228 closed May 30, 2024

3 Issues opened by 3 people

• Cannot convert neox model to HF

  #1231 opened May 28, 2024

• How to set the ffn hidden size parameter in gpt neox

  #1230 opened May 28, 2024

• The results of running eval show only 1 digit after decimal point for acc on all tested tasks

  #1227 opened May 22, 2024

5 Unresolved conversations

Sometimes conversations happen on old items that aren’t yet closed. Here is a list of all the Issues and Pull Requests with unresolved conversations.

• Add `intermediate_size` to GPT-NeoX models

  #1212 commented on Jun 19, 2024 • 2 new comments

• My servers used for multi-node training do not have ssh. How can I launch multi-node training using the torchrun command?

  #1203 commented on Jun 20, 2024 • 1 new comment

• Deepspeed benchmarking

  #878 commented on Jun 18, 2024 • 0 new comments

• Dmoe integration

  #1210 commented on May 22, 2024 • 0 new comments

• Add Transformer Engine

  #1213 commented on May 28, 2024 • 0 new comments