Building our Neural Network - Deep Learning and Neural Networks with Python and Pytorch p.3

The core work in this installment is building a complete feed-forward neural network in PyTorch: defining a model class, wiring fully connected layers, specifying how data flows through them, and producing class-probability outputs via log-softmax. The practical payoff is immediate—once the forward pass is in place, random “image-like” tensors can be pushed through the network to generate 10-class predictions, setting the stage for training in the next tutorial.

The model is implemented as a subclass of `nn.Module`, with an `__init__` method that constructs four linear layers. The input size is set to 784 because the network expects flattened MNIST-style images: 28×28 pixels are reshaped into a single vector of length 784. The hidden layers are configured as three stages of 64 neurons each (`784 → 64 → 64 → 64`), using `nn.Linear` for each fully connected transform. The final layer maps to 10 outputs (`64 → 10`), matching ten classes labeled 0 through 9.

A key PyTorch detail is the need to call `super().__init__()` during initialization. Omitting it triggers a cryptic error (“cannot assign module before module init call”), which the tutorial uses as a cautionary example of what goes wrong when the parent `nn.Module` initialization isn’t executed.

Data flow is defined in a `forward(self, X)` method. The input tensor is passed through `fc1`, `fc2`, and `fc3`, with a ReLU activation applied after each linear layer using `F.relu`. The output layer (`fc4`) is treated differently: it does not use ReLU. Instead, the network returns `F.log_softmax(X, dim=1)` to produce a log probability distribution over classes. The tutorial emphasizes that ReLU is appropriate for hidden layers to prevent values from exploding, while the output layer should be constrained to a probability-like interpretation suitable for multi-class classification.

To prove the wiring works, the tutorial generates random data shaped like a batch of 28×28 images. Passing a raw 28×28 tensor into the model causes a size mismatch because the network expects flattened vectors of length 784. The fix is reshaping with `view`, using `-1, 28*28` (or equivalent shapes like `1, 28, 28`) so the batch dimension is handled correctly. After reshaping, the model produces outputs for each of the 10 classes.

Finally, the tutorial notes that the network’s first passes are effectively untrained: weights aren’t meaningfully initialized yet, so predictions are not reliable. Still, the forward pass now returns something usable for later steps—computing loss and gradients—so the next stage can adjust weights to improve accuracy. It also highlights PyTorch’s flexibility: logic can be embedded inside `forward`, enabling more complex conditional architectures later, while gradients are handled automatically.