What is a neural network?

A neural network is a machine-learning model made of interconnected computational units, commonly called neurons, organized into layers. It learns a mapping from inputs to outputs by adjusting numerical parameters called weights and biases.

A simple feed-forward neural network can be visualized as:

Input layer        Hidden layer        Output layer

 x₁ ─────┐          ○ ─────┐
 x₂ ─────┼──────→   ○ ─────┼──────→   Prediction
 x₃ ─────┘          ○ ─────┘

What does a neuron do?

A neuron receives one or more input values. Each input is multiplied by a weight, the results are added together with a bias, and an activation function is applied.

z = w₁x₁ + w₂x₂ + ... + wₙxₙ + b

output = activation(z)

Where:

• x₁ ... xₙ are input values.
• w₁ ... wₙ are learned weights.
• b is a learned bias.
• activation is an activation function.

Weights determine how strongly different inputs affect the computation, while the bias allows the neuron to shift its response.

Layers in a neural network

A typical neural network contains three kinds of layers.

Input layer

The input layer receives the numerical representation of the data.

For example, a model predicting a house price might receive:

Area ─────────┐
Bedrooms ─────┼──→ Neural network
Age ──────────┤
Location ─────┘

Hidden layers

Hidden layers perform learned transformations of the data.

Input
  ↓
Hidden layer 1
  ↓
Hidden layer 2
  ↓
Learned representation

Networks with multiple hidden or computational layers are commonly associated with deep learning.

Output layer

The output layer produces the model's final result. Its structure depends on the task.

Examples include:

• A numerical value for regression.
• Scores or probabilities for classification.
• Multiple output values for multi-label or multi-output tasks.

How does a neural network learn?

Initially, many network parameters are initialized without encoding the final solution. Training repeatedly adjusts them based on data.

A simplified training cycle is:

Training example
      ↓
Forward pass
      ↓
Prediction
      ↓
Loss function
      ↓
Backpropagation
      ↓
Gradients
      ↓
Optimizer
      ↓
Updated weights and biases

The forward pass generates a prediction.

The loss function measures how well that prediction matches the training objective.

Backpropagation computes gradients of the loss with respect to the network parameters.

An optimizer, such as SGD or Adam, uses those gradients to update the parameters.

Repeating this process across many training examples allows the network to learn patterns useful for its task.

Why are activation functions important?

If every layer performed only linear transformations, stacking many such layers would still represent a linear transformation.

Nonlinear activation functions allow neural networks to represent much more complex functions.

Common activation functions include:

• ReLU
• Sigmoid
• Tanh
• GELU
• Softmax, commonly used at an output layer for multiclass probability distributions

Example

Suppose a network classifies an image as a cat or dog:

Image pixels
     ↓
Neural network layers
     ↓
Learned representations
     ↓
Output scores
     ↓
Cat: 0.92
Dog: 0.08

During training, the network adjusts its parameters so its outputs better match the correct labels according to the chosen loss function.

Neural networks and deep learning

A neural network does not necessarily have to be very deep. Deep learning generally refers to neural-network systems containing multiple learned layers that build increasingly useful representations.

Different neural-network architectures are designed for different kinds of problems, including convolutional neural networks, recurrent neural networks, and Transformers.

In short, a neural network is a layered mathematical model that learns from data by adjusting weights and biases so that its inputs are transformed into useful outputs.