Tensors in PyTorch | Video 2 | CampusX

Tensors sit at the center of deep learning in PyTorch because they turn real-world data—images, text, audio, video—into efficient, hardware-friendly arrays that can be processed with the same math at scale. The core takeaway is straightforward: a tensor is a generalized multi-dimensional array, and its shape (how many dimensions it spans) determines how neural networks compute forward passes, losses, gradients, and updates. That’s why learning tensors first isn’t optional; most deep learning work is essentially tensor manipulation and tensor math.

The lesson starts by defining tensors as specialized multi-dimensional arrays designed for mathematical and computational efficiency. Dimension is treated as the number of directions a tensor spreads across: a scalar is a 0D tensor (a single number), vectors are 1D tensors (like word embeddings), matrices are 2D tensors (like grayscale images), and 3D tensors represent color images with channels such as RGB. The explanation extends further: 4D tensors model batches of images (batch size plus image dimensions plus channels), and 5D tensors represent video data as sequences of frames across time, often batched for training.

Why tensors matter in practice comes down to three reasons. First, they make common neural-network operations efficient—addition, multiplication, dot products, and other element-wise or reduction computations. Second, they provide a uniform way to represent different modalities: images as number grids, text as vectors, and video as stacked frame data. Third, tensors enable fast computation on GPUs and TPUs through parallelism. A simplified example of element-wise matrix addition is used to illustrate the speed gap: CPU execution runs operations sequentially, while GPU execution can process many elements in parallel, producing large speedups for large tensors.

The walkthrough then shifts from concepts to PyTorch mechanics. It begins with setting up the environment using googleapis.com (including checking the installed PyTorch version and whether a GPU is available). It demonstrates basic tensor creation with functions like torch.empty (allocates memory without initializing values), torch.zeros (initializes to zero), torch.ones (initializes to one), torch.rand (random values), torch.tensor (from explicit Python data), torch.arange (range with steps), torch.linspace (linearly spaced values), torch.eye (identity matrix), and torch.full (fill with a constant). It also covers tensor metadata and correctness: retrieving shape via x.shape, copying shapes with methods like torch.zeros_like and torch.ones_like, and handling data types (dtype) explicitly—especially when random initialization needs floating-point outputs.

From there, the lesson catalogs key tensor operations: scalar operations with a tensor and a number, element-wise operations across tensors (addition, subtraction, multiplication, division, modulo), and reduction operations like sum, mean, median, max/min, argmax, and argmin. It also includes linear algebra operations such as matrix multiplication, dot products, transpose, determinant, inverse, and functions like log, exp, sqrt, sigmoid, softmax, and clamp. Finally, it covers practical “engineering” behaviors: in-place operations (using an underscore suffix like relu_), safe copying via clone (to avoid shared-memory bugs from assignment), moving tensors between CPU and GPU (using a device object and .to(device)), reshaping (reshape, flatten, permute, unsqueeze, squeeze), and converting between NumPy arrays and PyTorch tensors (torch.from_numpy and .numpy()). The result is a complete foundation for building and debugging neural networks, where nearly every step depends on getting tensor shapes, dtypes, and device placement right.