Fine-Tuning Large Language Models with Hugging Face

• Understanding what powers modern conversational AI systems, copilots, and intelligent assistants
• Large Language Models as transformer-based architectures trained on large-scale datasets
• How LLMs learn language patterns, context, and semantic relationships
• Common transformer architecture variants:
  • Encoder-only models for classification and understanding
  • Decoder-only models for text generation
  • Encoder-decoder models for translation and summarisation
• Overview of widely adopted model families, including BERT, T5, GPT, Llama, Claude, Gemini, and Mistral
• Using pre-trained transformers without additional training
• Zero-shot, one-shot, and few-shot prompt engineering
• Comparing the Generative AI lifecycle with the traditional machine learning lifecycle
• Common LLM limitations, including hallucination, bias, context-window constraints, and outdated knowledge

Hands-on

• Text summarisation using prompt engineering
• Grammar correction and content rewriting
• Comparing zero-shot and few-shot prompting results

• Pre-transformer approaches, including Word2Vec, recurrent neural networks, and Seq2Seq models
• Limitations of recurrent models and sequential processing
• Introduction to attention in Seq2Seq architectures
• Why Seq2Seq with attention is insufficient for large-scale language modelling
• Self-attention and query, key, and value representations
• Scaled dot-product attention
• Multi-head attention and parallel representation learning
• Core transformer building blocks:
  • Attention layers
  • Feed-forward neural networks
  • Residual connections
  • Layer normalization
• Encoder-only, decoder-only, and encoder-decoder architectures
• Positional encoding and token-order preservation
• Padding masks and causal attention masks

Hands-on

• Machine translation using Seq2Seq with attention
• Visualising attention scores across tokens
• Building a simplified transformer model from scratch
• Comparing recurrent and transformer-based models

• Understanding the role of data quality in LLM training, fine-tuning, retrieval, and evaluation
• Identifying and collecting domain-specific text data
• Working with structured, semi-structured, and unstructured data sources
• Data ingestion from documents, APIs, databases, repositories, and cloud storage
• Text extraction from PDF, HTML, JSON, CSV, and document formats
• Data cleaning and normalization:
  • Removing noise, markup, and corrupted records
  • Language detection and filtering
  • Handling missing and malformed content
• Duplicate and near-duplicate content detection
• Personally identifiable information and sensitive-data removal
• Data labelling strategies for supervised fine-tuning and evaluation
• Preparing instruction, response, conversational, and preference datasets
• Dataset balancing, sampling, splitting, and leakage prevention
• Token-length analysis and context-window management
• Data versioning, lineage, governance, and reproducibility
• Building reusable data pipelines for LLM applications

Hands-on

• Building a text ingestion and cleaning pipeline
• Detecting duplicate and low-quality records
• Converting raw data into instruction-response format
• Creating training, validation, and evaluation datasets

• Overview of the Hugging Face ecosystem
• Hugging Face Hub, Transformers, Datasets, Tokenizers, PEFT, TRL, and Accelerate
• Searching for and selecting suitable pre-trained models
• Understanding model cards, dataset cards, licences, benchmarks, and restrictions
• Loading, exploring, filtering, and transforming datasets
• Tokenisation, vocabulary, padding, truncation, and special tokens
• Loading models for classification, summarisation, translation, and generation
• Using Hugging Face pipelines for rapid experimentation
• Configuring models and tokenizers
• Managing checkpoints and model versions
• Saving and reloading trained models and adapters
• Basic model and dataset sharing through the Hugging Face Hub

Hands-on

• Loading and analysing a Hugging Face dataset
• Tokenising text for classification and generation
• Using pre-trained models for common NLP tasks
• Saving and restoring model checkpoints

• Understanding why inference-time techniques are often preferred over full model fine-tuning
• Prompt engineering as an application-development technique
• Creating clear system prompts, task instructions, constraints, and output formats
• Zero-shot, few-shot, and example-based prompting
• Chain-of-thought-style decomposition and structured reasoning workflows
• Prompt chaining for multi-step business processes
• Generating structured outputs using JSON schemas and validation rules
• Tool use and function calling for external systems
• Retrieval-Augmented Generation for grounding model responses
• Query rewriting, retrieval, reranking, and context construction
• Managing context windows and selecting relevant information
• Conversation history and memory-management strategies
• Generation controls:
  • Temperature
  • Top-p and top-k
  • Maximum tokens
  • Stop sequences
  • Frequency and repetition penalties
• Self-consistency and multi-response selection
• Output validation, grounding checks, and retry mechanisms
• Prompt caching, response caching, and semantic caching
• Choosing between prompting, RAG, tools, agents, and fine-tuning

Hands-on

• Building a structured-output extraction workflow
• Creating a multi-step prompt chain
• Implementing a basic Retrieval-Augmented Generation pipeline
• Connecting an LLM to an external tool or function
• Comparing prompting, retrieval, and fine-tuning for the same task

• Deciding when to prompt, retrieve, fine-tune, or train a model
• Difference between:
  • Pre-training
  • Continued pre-training
  • Instruction tuning
  • Task-specific fine-tuning
• Computational and infrastructure requirements for LLM training
• GPU memory, batch size, sequence length, and training-time considerations
• Full fine-tuning and its impact on cost and scalability
• Catastrophic forgetting and loss of general model capabilities
• Single-task versus multi-task fine-tuning
• Transfer learning for domain-specific applications
• Dataset preparation and train-validation-test splitting
• Loss functions, optimizers, learning rates, and training schedules
• Gradient accumulation, mixed-precision training, and checkpointing
• Detecting overfitting, underfitting, and data leakage
• Experiment tracking and model reproducibility

Hands-on

• Full fine-tuning of a sequence-to-sequence model
• Analysing catastrophic forgetting
• Sentiment analysis using transfer learning
• Comparing baseline and fine-tuned model performance

• Motivation for Parameter-Efficient Fine-Tuning
• Comparing PEFT with full model fine-tuning
• Frozen model weights and trainable adapter parameters
• LoRA architecture and low-rank updates
• QLoRA for memory-efficient fine-tuning of quantised models
• Prompt tuning, prefix tuning, and adapter-based methods
• Selecting PEFT techniques based on:
  • Model size
  • Available hardware
  • Dataset size
  • Task complexity
  • Deployment requirements
• Configuring LoRA rank, alpha, dropout, and target modules
• Saving, loading, merging, and reusing adapters
• Supporting multiple use cases through separate adapters
• Evaluating PEFT quality, efficiency, and operational cost

Hands-on

• Fine-tuning a transformer model using LoRA
• Applying PEFT for Named Entity Recognition
• Fine-tuning a model for text summarisation
• Comparing full fine-tuning with PEFT based on memory, time, and quality

• Understanding why traditional accuracy metrics are insufficient for generative AI
• Defining evaluation criteria based on business and technical requirements
• Evaluating different LLM capabilities:
  • Relevance
  • Correctness
  • Completeness
  • Coherence
  • Faithfulness
  • Safety
  • Robustness
• Evaluating classification tasks using accuracy, precision, recall, and F1 score
• Evaluating generation tasks using BLEU, ROUGE, METEOR, and BERTScore
• Evaluating summarisation, translation, extraction, and question-answering tasks
• Assessing hallucination and factual consistency
• Evaluating Retrieval-Augmented Generation:
  • Retrieval precision and recall
  • Context relevance
  • Groundedness
  • Faithfulness
  • Answer relevance
• Measuring latency, throughput, token usage, and cost per request
• Testing model robustness against prompt variations and adversarial inputs
• Comparing models across quality, cost, speed, safety, and deployment constraints

Hands-on

• Creating a domain-specific evaluation dataset
• Comparing multiple models on the same task
• Evaluating RAG responses for relevance, groundedness, and faithfulness
• Building an automated evaluation pipeline
• Producing a model comparison scorecard

• Introduction to model alignment and human preference learning
• High-level RLHF workflow:
  • Supervised fine-tuning
  • Reward model creation
  • Policy optimisation
• Role of human preference and ranked-response datasets
• Introduction to Proximal Policy Optimization
• Reward hacking, bias, scalability, and annotation challenges
• Overview of Direct Preference Optimization and related alternatives
• Understanding when organisations should use alignment techniques rather than standard fine-tuning

Hands-on

• Preparing a small preference dataset
• Comparing preferred and non-preferred model responses
• Demonstrating preference-based model optimisation using a lightweight example

• Infrastructure and cost challenges in LLM deployment
• Introduction to reduced-precision inference
• Comparing FP32, FP16, BF16, INT8, and INT4 formats
• Post-training quantisation and quantisation-aware training
• Impact of quantisation on quality, memory, latency, and throughput
• Weight pruning and parameter reduction
• Knowledge distillation for producing smaller models
• Model-serving patterns:
  • Online inference
  • Batch inference
  • Dynamic batching
  • Streaming
  • Asynchronous processing
• Managed APIs versus self-hosted models
• CPU, GPU, and specialised accelerator considerations
• Containerisation, inference endpoints, autoscaling, and load balancing
• Prompt caching, model caching, and request batching
• Monitoring:
  • Latency
  • Throughput
  • Token consumption
  • Error rates
  • Output quality
  • Infrastructure cost
• Model versioning, canary releases, rollback, and failure recovery
• Prompt injection, data leakage, insecure output handling, and model abuse
• Input validation, output filtering, access control, and safety guardrails

Hands-on

• Quantising a pre-trained language model
• Measuring model size, memory consumption, latency, and output quality
• Serving an optimised model through an inference API
• Comparing managed API and self-hosted deployment performance

End-to-End Project: Toxic Content Detection Using Model Distillation

Participants will design, evaluate, optimise, and deploy a transformer-based solution for identifying toxic or harmful content.

Project Activities

• Define the business problem and model success criteria
• Ingest, clean, label, and version the dataset
• Perform class-distribution and data-quality analysis
• Select an appropriate teacher transformer model
• Fine-tune or adapt the teacher model
• Distil the teacher model into a smaller student model
• Apply PEFT, quantisation, or pruning where appropriate
• Create a benchmark dataset and evaluation framework
• Evaluate accuracy, precision, recall, F1 score, latency, throughput, and model size
• Test robustness against spelling variations and adversarial content
• Compare teacher and student models
• Build an inference API or application workflow
• Add monitoring, logging, safety controls, and model-version management
• Optimise the solution for cost, performance, and production readiness

• End-to-end solution demonstration
• Data pipeline and architecture review
• Model and technique selection justification
• Evaluation scorecard presentation
• Performance and cost comparison
• Discussion of scalability, security, and operational trade-offs
• Recommendations for production deployment and future improvements