Chapter 5: Neural Networks and Deep Learning (Part 1)

From Biological Brains to Artificial Nodes

Deep learning takes its structural inspiration from human neurobiology. We examine the biological neuron, noting how dendrites receive signals, the cell body processes them, and axons transmit electrical impulses across synapses. We contrast the sequential architecture of traditional computers (where a CPU fetches data from separate RAM) with the massively parallel, distributed architecture of neural networks, where memory and computation are integrated across interconnected nodes.

This concept of distributed processing can be understood by comparing a single centralized CPU to a decentralized village Panchayat (council). In a centralized system, one single unit makes all decisions sequentially. In a Panchayat, however, decision-making is distributed: every elder acts as an interconnected node, independently weighing different pieces of evidence and community feedback simultaneously. Their individual assessments are combined, and only through the weighted consensus of the entire network is a balanced resolution achieved.

Network Architecture and Backpropagation

We construct artificial neural networks step-by-step, organizing artificial neurons into structured layers:

• Input Layer: Receives raw numerical features from the external world.
• Hidden Layers: Intermediate layers where linear combinations of inputs are transformed by mathematical Activation Functions (such as the Sigmoid curve, Step functions, or ReLU) to capture non-linear relationships.
• Output Layer: Produces the final classification or prediction.

To train these Multilayer Perceptrons, we introduce the Backpropagation Algorithm, pioneered mathematically by Seppo Linnainmaa. When a network makes a prediction, the error is calculated at the output layer and transmitted backward through the network. As the error signal flows backward, the calculus of chain-rule differentiation adjusts the weight of every connection, gradually minimizing mistakes.

This backward process of learning from mistakes is identical to the craft of a traditional pottery artisan (kumhar) shaping clay on a wheel. When a developing vessel shows a slight wobble or imperfection at the rim, the craftsman does not discard the piece; instead, he traces the cause of the error backward from the top edge down through the walls to the pressure of his thumbs at the base. By continuously adjusting his hand pressure and technique backward from the observed flaw, he gradually eliminates mistakes until the vessel achieves perfect balance and form.