Parameter Efficient Fine Tuning

Parameter-efficient fine-tuning is presented as a practical way to adapt large Transformer and language models to new tasks without retraining the full weight set. Instead of updating every parameter with gradient descent, methods insert small, task-specific components—adapters, low-rank updates, or learned “prefix” key/value states—while freezing the original model. The payoff is lower storage and faster deployment of task specializations, since each new task can be shipped as a small parameter bundle rather than a full model checkpoint.

The baseline concept of fine-tuning is laid out first: start from a pre-trained model, change the output head if the label space differs, and then train with gradient descent while freezing some layers (often the entire backbone at first, then gradually unfreezing). Full fine-tuning yields a new complete parameter set, but it is expensive in both compute and—crucially for many real-world deployments—model distribution and storage.

The talk then pivots to the 2019 adapter approach (Transformers adapters). Adapters are small bottleneck networks inserted into Transformer blocks (commonly around the feed-forward sublayer, sometimes also near attention). They include a skip connection so the adapter can behave like an identity mapping when needed, which helps preserve the original model’s behavior. During training, only the adapter parameters are updated; the rest of the Transformer remains frozen. Experiments on GLUE benchmark tasks are used to argue that adapter tuning can reach performance comparable to full fine-tuning while training far fewer parameters. The discussion emphasizes that the comparison is often framed in terms of parameter count (storage/transfer) more than raw compute, since the frozen backbone still runs during forward passes.

From there, the ecosystem expands. AdapterHub is described as a standardized way to publish and reuse these task-specific adapter modules, built by adding “hooks” into the Hugging Face Transformers library so adapters can be inserted into the residual stream. The talk also surveys several parameter-efficient alternatives:

LoRA (low-rank adaptation) decomposes weight updates into low-rank matrices, reducing the number of trainable parameters while still targeting key transformations (originally discussed for attention projections, but viewed as broadly applicable). Prefix tuning is framed as a different mechanism: rather than inserting a new module into the network, it learns extra “virtual” past key/value states that are prepended to the attention cache. These learned key/value vectors steer generation for a task while leaving the Transformer weights untouched. Prompt tuning and P-tuning are positioned as related variants that place learnable embeddings earlier in the pipeline (prompt/prefix learning), with P-tuning using an LSTM-based prompt encoder to generate task-specific virtual tokens.

The session closes with additional lightweight strategies such as IA3-style modulation, which adjusts intermediate activations using small learned scaling parameters instead of adding new sub-networks. Throughout, the central theme is modular specialization: large models can be adapted to many tasks by shipping small, composable parameter sets—improving practicality for multi-task systems where loading a full fine-tuned model per task would be too costly.