* optimize data preprocessing

semaphore is a little too small and slows down tokenizing

* Make killall.sh less bruteforce

* [temporary] fix to index errors

* [temporary] fix to index errors

* print sizes of tensors when inspecting checkpoint (#382)

Co-authored-by: Samuel Weinbach <[email protected]>

* Use lru_cache for GPT2Tokenizer.bpe (#383)

GPT2Tokenizer currently uses an unbounded cache, which causes very
high memory usage with tools/preprocess_data.py

* Fix bug with number of evaluation steps (#384)

we were running way to many evaluation steps if the model is pipe parallel + has g.a.s on because of this line

```python
            for _ in range(neox_args.gradient_accumulation_steps):
```

- fixing this to 1 if the model is pipe parallel fixes the issue, as .eval_batch() already takes gradient accumulation steps into account.

* Create CITATION.cff

* Update CITATION.cff

* Update documentation (#392)

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* add info about installing fused kernels

* Update README.md

* Update README.md

* sparsity + minor typos

add the instructions to install triton

* change path to ssd-1

* typo

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

* Update README.md

Co-authored-by: Shivanshu Purohit <[email protected]>

Co-authored-by: Stella Biderman <[email protected]>
Co-authored-by: Samuel Weinbach <[email protected]>
Co-authored-by: Samuel Weinbach <[email protected]>
Co-authored-by: iczero <[email protected]>
Co-authored-by: Shivanshu Purohit <[email protected]>