DeeperSpeed 2.0 is based off upstream DeepSpeed 0.8.3 and is intended to closely match the upstream along with:

Any additional fixes we find that are yet to be accepted into DeepSpeed
Branches for optimal performance on EleutherAI's compute providers (e.g. stability, coreweave)

DeeperSpeed 1.0 is the legacy version based on DeepSpeed 0.3.15. This version is meant to be combined with GPT-NeoX v1.0.