What is a deep neural network?

A deep neural network (DNN) is a neural network with multiple learned computational layers between its input and output. These layers progressively transform input data into representations useful for the task being learned.

Basic structure

Input
  ↓
Hidden Layer 1
  ↓
Hidden Layer 2
  ↓
Hidden Layer 3
  ↓
...
  ↓
Output

The presence of multiple learned layers gives the network its depth.

How does a deep neural network work?

Each layer receives values from the previous layer and performs a learned transformation. For a fully connected layer, a simplified computation is:

h = f(Wx + b)

Here, x is the layer input, W contains learned weights, b contains learned biases, f is an activation function, and h is the resulting representation.

With several layers, these transformations are composed:

h1 = f(W1x + b1)
h2 = f(W2h1 + b2)
h3 = f(W3h2 + b3)

This composition allows deep networks to represent complex nonlinear functions.

Why are multiple layers useful?

Successive layers can build increasingly task-relevant representations. In an image model, a simplified intuition is:

Pixels
  ↓
Simple visual patterns
  ↓
Textures and shapes
  ↓
More complex structures
  ↓
Task-specific representation
  ↓
Prediction

The actual learned representations depend on the architecture, data, and training objective and do not necessarily correspond neatly to human-defined concepts.

Deep vs. shallow neural networks

A shallow neural network has relatively few learned layers, while a deep neural network contains more stacked layers.

Shallow:
Input → Hidden Layer → Output

Deep:
Input → Hidden 1 → Hidden 2 → Hidden 3 → ... → Output

There is no universal number of layers at which a network officially becomes "deep." The term generally describes neural networks with multiple layers of learned transformations.

How is a DNN trained?

Training commonly follows this cycle:

Input data
    ↓
Forward pass
    ↓
Prediction
    ↓
Loss calculation
    ↓
Backpropagation
    ↓
Gradients
    ↓
Optimizer
    ↓
Parameter update
    ↺

The forward pass produces a prediction. A loss function measures performance against the training objective. Backpropagation computes gradients, and an optimizer such as SGD or Adam uses those gradients to update trainable parameters.

Example

For multiclass image classification:

Image
  ↓
Deep neural network
  ↓
Learned representation
  ↓
Class scores
  ↓
Cat: 0.91
Dog: 0.06
Bird: 0.03

The network learns its parameters from training data rather than requiring developers to manually define rules for every class.

Are CNNs and Transformers deep neural networks?

They can be. Deep neural network is a broad description rather than one specific architecture. Deep architectures can include multilayer perceptrons, CNNs, RNNs, Transformers, autoencoders, and graph neural networks.

Advantages and challenges

Deep neural networks can learn complex nonlinear relationships and useful representations from high-dimensional data such as images, audio, and text. However, larger and deeper networks can require substantial computation and memory, can overfit, and can be difficult to optimize and interpret.

Techniques such as appropriate initialization, normalization, residual connections, regularization, and suitable optimization methods can help train deep networks effectively.

In short, a deep neural network is a neural network with multiple learned layers that progressively transform data into representations used to produce an output.