The process of creating LongForm from corpus examples. After collecting diverse examples from corpora, LLMs generate relevant instructions through zero-shot prompting in various styles. The final instruction is generated by selecting one of three styles (instruction (as in this case), informal chatbot, search engine query) and optionally including length information.

The distribution of the LongForm dataset in terms of the source of examples is below. It contains examples generated from raw text corpora via LLMs, structured corpus examples, as well as various NLP task examples such as email writing, grammar error correction, story/poem generation, and text summarization.

Most common verb and noun/auxiliary verb pairs of generated instructions from the corpora subset of the LongForm dataset.