Loading in your own data - Deep Learning basics with Python, TensorFlow and Keras p.2

A practical pipeline for turning the Microsoft “cats and dogs” image dataset into training-ready tensors is the core takeaway, including how to load images from disk, standardize them, assign numeric labels, and prepare shuffled feature/label arrays for a neural network. The workflow matters because deep learning models can’t learn from raw, variably sized images or string labels; they need consistent input shapes and numeric targets.

The process starts with downloading and extracting the dataset, which is organized into two folders—cat and dog—each containing roughly 12,500 images. The code then imports common tooling: NumPy for array work, Matplotlib for quick visual checks, OS for directory traversal, and OpenCV (cv2) for image loading and preprocessing. A base data directory is set (the transcript uses a local path), and a categories list defines the two classes. Using nested loops, the script walks through each category folder and iterates over every image file name.

Each image is read with OpenCV and converted to grayscale. That choice is deliberate: grayscale reduces input size by removing color channels, and the transcript argues color isn’t essential for distinguishing cats from dogs. After conversion, the code visualizes a sample using Matplotlib to confirm the preprocessing worked as expected. It also inspects the grayscale array shape, noting that images vary in dimensions (some are landscape, portrait, or square). To make the dataset uniform, every image is resized to a fixed square size—set to 50×50 in the example—so the model receives consistent input dimensions.

Next comes dataset construction. An empty list collects training samples, and a helper loop assigns labels by mapping categories to numbers: dog becomes 0 and cat becomes 1 (the exact mapping is arbitrary as long as it’s consistent). For each image, the resized grayscale array is appended alongside its numeric label. The transcript mentions that some images can be broken and suggests using try/except to skip problematic files, though the example opts to pass.

Two key training-data hygiene steps follow. First, the dataset should be balanced so neither class dominates; otherwise the model can learn a misleading shortcut (e.g., always predicting the majority class). Second, the data must be shuffled. Without shuffling, the model would see all dogs first and then all cats, which can lead to unstable learning patterns.

Finally, the shuffled list is split into feature and label arrays: X holds image data and Y holds labels. X is converted into a NumPy array and reshaped to match the expected neural-network input format for grayscale images: (number_of_samples, image_size, image_size, 1). The transcript also flags that color images would require a different channel dimension (changing the final “1” to “3”). To avoid repeating expensive preprocessing every training run, the prepared X and Y are saved using pickle for later reuse. The next tutorial step is then to feed this compiled dataset into a convolutional neural network for classification and evaluate accuracy.