Research - An Introduction to Text Analysis with Transformers and LLMs in Python
Transformers and Large Language Models (LLMs) are widely used in modern text analysis, including text classification, text generation, summarisation, translation, and chatbots. In this course, you will learn how Transformers and LLMs work and how to apply them practically using Python. Hands-on examples will range from everyday prompting to two key research workflows: benchmarking different models to compare their performance, and fine-tuning a model to build a specialised assistant with reliable evaluation.
Objectives
Acquire the key competencies needed to use LLMs for text classification, text generation, and effective chatbot usage, including prompt engineering as well as benchmarking and fine-tuning workflows to support research applications.
Target audience
Any PhD students, post-docs, researchers of UNIL who would like to use LLMs in their research
Content
At the end of the course, the participants are expected to:
- Understand how Transformers and LLMs work
- Be able to use LLMs for text classification, text generation and effective chatbot usage in Python
- Master prompt engineering tailored for researchers
- Benchmark and compare multiple models on a defined task using a reproducible evaluation protocol
- Understand the basics of fine-tuning and how to evaluate whether a fine-tuned model improves performance for a specific application
Length
1 day
Organization
Once per year
Location
In-person only (no online option)
Practicals
Prerequisites
- Basic knowledge of deep learning: we assume that you know how simple feedforward neural networks work, including how to interpret accuracy and loss curves (for example by attending the course "A Gentle Introduction to Deep Learning with Python and R").
- Be comfortable with Python programming
IMPORTANT: To do the practicals
- On UNIL JupyterLab: Access requires that you connect either via the eduroam Wi-Fi with your UNIL account or through the UNIL VPN. This point is especially crucial for researchers from the CHUV.
- On your laptop: No account requirement
- On Curnagl: Please register using your UNIL email address
- Note that in all cases you need to bring your own laptop