High-Frequency Trading - Hard-Rule & Supervised Traders - Implemented (4)

The core takeaway is how the trading dataset is engineered for both supervised learning and “hard-rule” trading: each day’s limit-order-book (LOB) stream is converted into a walk-forward, nine-fold training/test structure with 144 carefully designed features and five future prediction horizons. That setup matters because it turns irregular, event-driven market data into consistent supervised targets—so models can learn when the mid price is likely to rise, fall, or stay flat after accounting for transaction costs.

Data preparation starts by dropping weekends: Sunday and Saturday are removed, leaving trading days that follow a consistent pattern. Each remaining day is split into nine consecutive folds arranged in a walk-forward manner. Training grows iteratively: a model is trained on earlier folds, tested on the next subset, then retrained as additional folds become available. Metrics are computed by averaging performance across all test subsets, using a dataset size of 400,000 samples with 144 features.

The feature set is built from raw 10-level LOB snapshots—ask and bid prices and volumes across 10 depth levels—plus derived indicators that summarize both “state” and “movement.” State features include spread and mid price (best ask minus best bid, and their average), total and level volumes, and average prices. For each depth level, the pipeline computes normalized price differences relative to the best ask/bid and normalizes volumes. It also adds time-sensitive measures: differences between previous and current prices or volumes divided by the elapsed time between events, plus “intensity” indicators that capture how quickly price changes. The result is a mix of continuous and categorical-style signals (including thresholded indicators).

Targets are defined around the mid price at future horizons. Five separate horizons are generated, ranging from near-term to more distant future points (described as “next one, two, three, five, or ten events per millisecond,” with the exact construction tied to event timing). Each target label is categorical direction rather than a raw price value: the dataset computes the percentage change of the mid price relative to the current mid price, then assigns labels based on a threshold of 0.002. Changes ≥ +0.002 become “up” (label 1), changes between −0.002 and +0.002 become “flat” (label 2), and changes ≤ −0.002 become “down” (label 3).

The trading logic connects these predictions to execution and fees. A strategy buys or goes short only if the predicted mid-price move is large enough to cover transaction costs and still deliver profit. The hard-rule baseline is simple: buy one share at 10:30 each day and sell at 6 p.m. the same day (then repeat daily). For the supervised approach, the plan is to adapt a random forest classifier (using scikit-learn) trained on the prepared features and five horizon-specific targets, then evaluate results by computing a “heat ratio” and saving outputs to CSV.

Overall, the dataset engineering—walk-forward splits, 10-level LOB feature construction, five horizon labels, and fee-aware decision framing—turns raw LOB streams into a supervised learning problem that can be implemented with a straightforward random forest pipeline and compared against a deterministic hard-rule trader.