Training Convnet - Deep Learning and Neural Networks with Python and Pytorch p.6

Convolutional neural networks can be trained end-to-end in PyTorch by building the model architecture, working out the “flattened” size between convolution and fully connected layers, and then running a basic training loop with an optimizer and loss function. The practical bottleneck in this workflow is not the training itself—it’s determining the correct input dimension for the first linear layer after multiple Conv2D and max-pooling operations, because PyTorch doesn’t provide a simple “flatten layer” that can be queried for shape the way some other frameworks make it feel.

The tutorial starts by defining a custom neural network class that inherits from nn.Module. It constructs three 2D convolution layers (Conv2d) with a 5×5 kernel and increasing channel depth: the first maps 32 channels to 32 convolutional features, then subsequent layers expand to 64 and 128 channels. After each convolution, max pooling with a 2×2 window and a ReLU activation are applied to reduce spatial dimensions while keeping the model expressive. This convolutional stack produces a tensor that still has height and width, so it must be reshaped before feeding into linear layers.

To handle that reshape, the code performs a “dummy forward pass” during initialization. A random input tensor shaped like the dataset images (batch size 1, channels 1, height 50, width 50) is passed through the convolution+pooling pipeline to measure the resulting output shape. The flattened dimension is then computed from that shape and used to set the input size of the first fully connected layer (FC1). The second linear layer (FC2) outputs two logits, matching a binary classification setup (e.g., cat vs. dog). The forward method then runs the same convolution/pooling stack, flattens the result with view, applies FC1 with ReLU, and finally returns FC2 outputs.

Once the model is defined, training begins with standard PyTorch components: an Adam optimizer with learning rate 0.001 and mean squared error (MSE) loss. The dataset is converted into tensors, image pixel values are scaled from 0–255 down to 0–1 by dividing by 255, and the data is split into training and validation sets using a 10% validation fraction. Training uses mini-batches (batch size 100) and a single epoch on CPU, iterating through slices of the training tensors. Each step zeroes gradients, computes outputs, calculates loss, backpropagates with loss.backward(), and updates parameters via optimizer.step().

Evaluation runs with torch.no_grad() and computes accuracy by taking argmax over predicted outputs and comparing to ground-truth labels. The reported accuracy is around 64%, better than random guessing but not yet strong. The tutorial closes by emphasizing that CPU training is slow and that the next step is moving the same code to GPU for faster experimentation, plus adding more epochs and better model analysis to decide when to stop training and how to compare architectures.