Large Language Models (LLMs) are revolutionising social science research with their exceptional text comprehension capabilities, opening new avenues for innovation. However, LLMs can introduce social biases into the output, and the reliability of their outputs can vary. Safely and responsibly harnessing LLMs while fostering innovation requires specialised training—a challenging task given the rapid pace of LLM research, with new models and updates emerging weekly.

Oxford LLMs addresses these challenges by uniting machine learning scientists, leading researchers, and industry practitioners to offer comprehensive training to social scientists. Our goal is to bridge the gap between academia and industry, as well as between social sciences and computer science, with a focus on natural language processing. Our workshop equips participants with the latest skills and fosters a community that promotes innovation in social science research, ensuring the safe and responsible use of LLMs.

• Ilya Boytsov, Coding Seminars Leader, NLP Lead ar Wayfair, Content Intelligence Team.
• Elena Voita, Lecturer, Research Scientist at FAIR (Meta AI), lecturer at Oxford LLMs 2023
• Maksim Zubok, Workshop organiser, DPhil candidate in Politics at Oxford University, Nuffield College.

The following lecture materials were and created by Elena Voita.

• The Evolutionary Journey of NLP from rule-based systems to modern Transformers-based models, which are the core technology underpinning LLMs. Video coming soon!
• Bias in LLMs and a (bit of) Interpretability. Video coming soon!
• LLMs and Alignment Video coming soon!

The following workshop materials were designed and implemented by Ilya Boytsov. We will upload the workshop recordings soon!

• Google Colab environment setup, general intro
• Introduction to Huggingface transformers library
• Topic modelling with Transformers using BERTopic library
• A guide how to fine-tune pretrained model for a classification task
• Parameter Efficient Fine Tuning (PEFT)
• Transformers interpretability, Attention visualisation, and saliancy methods (e.g. Integrated gradients)
• Model analys with classic NLP using Spacy
• Prompts and instructions with Llama 2
• Detoxifying summarisation model with Reinforcement Learning from Human Feedback (RLHF)

📧 For inquiries, contact [email protected]