This repo is to accompany a session run as part of the Mastering LLMs: A Conference For Developers & Data Scientists conference. The session focused on some of the data issues related to fine-tuning LLMs.

The goals of the notebooks are focused on balancing the requirement to have sufficiently diverse data, with high quality and the right quantity i.e. avoid duplication.

• Creating, curating, and cleaning data for LLMs

• 01_eda_and_deduplication
• 02-data-checks
• 03-synthetic-data-generation

• dataset-card-summaries: This folder contains a pipeline for generating a synthetic dataset focused on generating tl;dr summaries of datasets based on their dataset card.

• awesome-synthetic-datasets