Day 1 Monday, 4 November 2024
| 08:45 | Join in |
| 09:00 | Welcome, Motivation & Introduction |
| ===== | Introduction to LLMs |
| 09:15 | Overview |
| 09:45 | Huggingface Ecosystem |
| 10:00 | Transformer Anatomy |
| 10:15 | Tokenization & Embeddings |
| 10:30 | -- BREAK -- |
| 10:45 | Hands-on example |
| ===== | Memory-efficient Training |
| 11:15 | Quantization |
| 11:30 | PEFT |
| 11:45 | unsloth |
| 12:00 | Hands-on example |
| 12:30 | Q & A |
| 13:00 | End of day 1 |
Day 2 Tuesday, 5 November 2024
| 08:45 | Join in |
| 09:00 | Recap |
| ===== | Distributed Training |
| 09:15 | SLURM |
| 09:30 | DDP |
| 10:00 | Huggingface Accelerate |
| 10:30 | -- BREAK -- |
| 10:45 | ZeRO & DeepSpeed, FSDP |
| 11:15 | Hands-on examples |
| ===== | Evaluation |
| 11:45 | Methods & Metrics |
| 12:00 | Monitoring |
| 12:15 | Inference |
| 12:30 | Hands-on example |
| 12:50 | Wrap-up |
| 13:00 | End of day 2 |
