Building an LLM fine-tuning Dataset

Fine-tuning an LLM on Reddit comments is less about model training and more about building a usable dataset—especially when the goal is multi-turn, multi-speaker conversations. The workflow centers on extracting large volumes of Reddit comment data (not posts), converting it into conversation chains, and then formatting those chains into training samples that a causal language model can learn from. The payoff is a dataset tailored to realistic social dynamics: on Reddit, responses often involve more than two participants, and the “bot” reply needs to be grounded in a history of prior comments.

The process starts with sourcing comment archives. A long-running Reddit comment dataset exists via torrents/archives and, crucially for scale and convenience, through a maintained BigQuery dataset containing Reddit comments from roughly 2005 through the end of 2019. That BigQuery resource is large enough to train or fine-tune models at meaningful sizes (the transcript mentions a 7B-class model as a plausible target) and to filter behavior by subreddit. The practical bottleneck is not the amount of data, but exporting it in manageable chunks, downloading it reliably, and reshaping it into something trainable.

To handle export, the workflow uses BigQuery exports to Google Cloud Storage, typically organized by year and month. The transcript emphasizes that CSV exports become unwieldy for comment bodies, so JSON is preferred, even though it creates many small files. Downloading and storage become operational problems: compression (gzip) can reduce transfer size but adds decompression overhead, and unstable internet can force overnight downloads. Because decompression and parsing are expensive at scale, the workflow pre-decompresses files locally (or on a NAS) to avoid repeating CPU work during later iterations.

Once the raw JSON is decompressed, the next step is data wrangling: selecting only the fields needed for conversation reconstruction (comment body, author, timestamps, IDs, parent IDs, and score). The transcript notes that many comments won’t participate in reply chains, and that top-level posts (which may not have a parent) can be missing from naive “parent-based” reconstruction—so the dataset may need additional logic to anchor replies to titles. A key design decision is how to build training samples: instead of simple instruction-response pairs, the dataset is assembled as conversation chains by walking parent IDs backward until no parent is found, then packaging the resulting history.

For WallStreetBets, the dataset is filtered using minimum reply length and minimum upvotes (score), producing multiple dataset variants (e.g., min score 3/5/10 and min length 2/5). The transcript also explores multi-speaker formatting by assigning speaker IDs to each author and using a dedicated bot name (“WallStreetBot”) for the target reply. The resulting samples can be used either as full prompt+response training text or as structured instruction formats, depending on the fine-tuning approach.

On the training side, the workflow experiments with model choices and ultimately uses parameter-efficient fine-tuning (LoRA/QLoRA-style) with an “AutoPFT” approach for causal language models. That automation supports checkpointing adapters and testing them without repeatedly merging or dequantizing full models—making iteration faster and cheaper. Early results suggest the curated, filtered dataset improves quality versus an earlier attempt, and the transcript claims diminishing returns beyond roughly 500–1,000 training steps, with the possibility that far fewer samples could work if the filtering is strong enough. The end state is a set of Hugging Face datasets and fine-tuned adapters, plus a plan to iterate on better prompts and more targeted sample selection (including higher-score thresholds) while avoiding “junk” comments that inflate scores without meaningful content.