Fine-tuning Llama 3.2 on Your Data with a single GPU | Training LLM for Sentiment Analysis

Fine-tuning Llama 3.2 (1B) for sentiment classification on a custom mental-health dataset can jump accuracy from roughly 30% to nearly 85% using a single GPU. The workflow builds a labeled dataset from Hugging Face, filters it to fit GPU memory constraints, converts each example into an instruction-style prompt that forces the model to output exactly one category, then applies LoRA-style fine-tuning via TorchTune with quantization-aware options available. After training for one epoch, the tuned model’s predictions stay within the allowed class list and the classification report improves across categories.

The process starts with TorchTune—described as a PyTorch-team library with ready configurations for many LLMs, including Llama variants, and support for single-device fine-tuning. The dataset contains about 53k rows from a Hugging Face repository focused on sentiment analysis for mental health. Each row pairs a text “statement” with a single label (“status”) drawn from seven categories, with a highly skewed class distribution. To reduce memory pressure during tokenization and training, the pipeline measures word counts (via counting spaces) and drops long examples—keeping 99.6% of data after removing statements over 1,000 words. It then caps examples per class (at most 5,000 per status) to rebalance the dataset, producing a smaller training set roughly half the original size.

For modeling, the approach uses an instruction-tuned Llama 3.2 1B model rather than the base model. The instruction prompt is explicit: “Classify this text for one of the categories… Choose from one of the category only… Reply only with the category,” followed by the statement and the list of valid labels. TorchTune requires input/output columns in a particular format, so the script generates JSON files for training and testing where the input is the prompt and the output is the normalized category label.

Before fine-tuning, the base instruct model is evaluated using the same prompt template and generation settings (including a token limit). Predictions often include formatting artifacts or labels outside the allowed class names, so the evaluation cleans outputs by stripping anything not in the class list. The resulting classification report is weak—around 30% accuracy—reflecting both the difficulty of the task (even humans can disagree on label assignment) and the model’s limited instruction-following for this specific label set.

Fine-tuning then uses a TorchTune configuration adapted from the LoRA/“war” setup, targeting attention modules with LoRA parameters such as rank and alpha (with zero dropout in the described config). Training runs on a single device (an NVIDIA L4 with 24GB VRAM is mentioned, with T4 as a possible alternative if batch size is reduced). The batch size is set to 4, and optimization uses AdamW with a cosine learning-rate schedule and warmup of 100 steps. Training takes about 35 minutes for roughly 20,000 examples on the L4, and metrics show loss behavior consistent with learning.

Afterward, the tuned checkpoint is packaged into a Hugging Face repository and evaluated with the same prompt and generation pipeline. The tuned model’s outputs remain within the valid categories, and accuracy rises to nearly 85%. The classification report improves across categories, with notable gains in precision for depression and personality disorder, and overall improvements in precision/recall/F1 scores for every label. The takeaway is that when the task is tightly defined (single-label output from a fixed set) and the dataset is curated to fit hardware limits, even a small 1B model can deliver large gains from lightweight fine-tuning on one GPU.