Incorporation of LION Optimizer in GPT-NeoX

[withwsf]

I am writing to request the integration of Google's LION optimizer(https://github.com/lucidrains/lion-pytorch) within the GPT-NeoX framework. This feature would potentially enhance training efficiency and performance for large-scale models. While I understand a manual adaptation is feasible, an official, project-level integration would benefit the wider community and ensure optimal compatibility and performance.