High-Frequency Trading Data and Features - Explained (3)

High-frequency trading data is built around a limit order book that updates on extremely short timescales—nanoseconds to microseconds—so the dataset’s core job is to capture how buy and sell orders stack up at different price levels and how those stacks evolve. Instead of trading “hours, minutes, or seconds,” the market state changes so quickly that models must treat the order book as a dynamic, time-ordered sequence of snapshots. Each snapshot records the best bid and best ask (the closest buy and sell price levels) plus the depth of liquidity across multiple levels, enabling features like spread and level-wise volumes that reflect real trading conditions.

A limit order book is an electronic system where all traders submit orders through a network and the exchange matches them. Orders arrive as buy orders (from traders seeking to purchase) and sell orders (from traders seeking to sell). When a buy order is placed at a price that matches an existing sell order at the same price, a transaction occurs and the corresponding order quantities are removed from the relevant levels—making the book change immediately. The book is represented as price “levels” on both sides: each level corresponds to a price and aggregates the total quantity resting at that price. The “best bid” is the highest buy price currently requested; the “best ask” is the lowest sell price currently requested. The spread—often the most informative feature—is computed between these closest levels (best ask minus best bid), though spreads between other level pairs can also be derived.

Because the order book is stochastic and “spiky,” the dataset is normalized but normalization choices strongly affect downstream results. Min-max scaling, score-based scaling, and other methods are available, and selecting an appropriate scaler matters: if extremes dominate, the remaining structure can become less meaningful. Outlier handling is also recommended, since the distributional quirks of limit order books can distort learning.

The dataset is organized as post-processed event streams for five Nasdaq instruments (one file per stock per day), reconstructed from exchange messages such as order submissions, cancellations, and executions. Network effects can cause events to arrive out of order, so the pipeline reorders them and then generates future targets from the reconstructed market state. Snapshots form a time series: the time between events is irregular because the market only records states when events occur, so consecutive snapshots can be separated by anything from milliseconds to minutes. The data spans 10 days with nearly 4 million consecutive samples, and it includes stratified, walk-forward splits for supervised learning targets across nine folds.

Targets are provided for five classification tasks, while reinforcement learning can use the raw data directly. For modeling, the book is recorded with 10 levels per snapshot (five bid levels and five ask levels), even though real books contain more depth. Finally, the dataset includes trading-session metadata and recommends trimming anomalous periods—cutting roughly the first and last half hour after market open/close in Helsinki time—so models learn from typical regimes rather than edge-case behavior. Prices are discretized using tick size (e.g., one cent) and scaled (multiplied by 10,000) to avoid floating-point storage issues, and each message includes nanosecond timestamps, price levels, quantities, and event types (submission, cancellation, execution).