Is Support FIM mode?

[zxyscz]

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

Describe the solution you'd like
A clear and concise description of what you want to happen.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

[ncoop57]

it was moved to a separate branch: https://github.com/EleutherAI/gpt-neox/blob/FIM-clean