Deep Learning
Deep learning is a branch of machine learning that uses neural networks with many layers. These layers learn representations directly from data, making deep learning especially useful for images, language, audio, and multimodal systems.
What is inference in deep learning?
Inference in deep learning is the process of using a trained neural network to make predictions or generate outputs from new, unseen input data. Training is when the model learns. Inference is when the trained model is actually used. How inference works A simplified flow is: New input → Trained model → Prediction or generated output For example: An image classifier receives a new image and predicts its class. A speech model receives audio and produces text. A generative model receives a prompt and produces an output. What happens during inference? During inference: The model's learned weights are typically fixed. A forward pass is performed through the network. No backpropagation is needed. Gradients are usually disabled because the model is not being trained. This generally makes inference less computationally expensive than training. Why does inference performance matter? In production systems, inference often needs to be fast and efficient. Important factors can include: Latency Throughput Memory usage Model size Hardware efficiency Techniques such as quantization, pruning, batching, and specialized accelerators can be used to improve inference efficiency. Is inference always deterministic? No. Some models produce the same output for the same input, while generative models may use sampling and therefore produce different outputs across runs. The next article explains the distinction in more detail in “What is the difference between training and inference?” . In short, inference is the stage where a trained deep learning model uses what it has learned to make predictions or generate outputs for new inputs.
What is the difference between training and inference?
Training and inference are two different stages of using a deep learning model. Training is when the model learns from data, while inference is when the trained model uses what it learned to produce predictions or outputs. During training The model processes training examples, calculates how wrong its predictions are, and updates its weights. A simplified flow is: Input → Forward pass → Loss → Backpropagation → Weight update Training usually involves: Forward propagation Loss calculation Backpropagation Gradient computation Optimizer updates Repeated passes over training data Because gradients and parameter updates are required, training usually needs more memory and computation. During inference The model receives new input and performs a forward pass using the learned weights. A simplified flow is: New input → Trained model → Prediction or generated output During inference: Weights are normally not updated. Backpropagation is not required. Gradients are usually disabled. The focus is on fast and efficient prediction. Why can model behavior differ between training and inference? Some layers behave differently depending on the stage. For example: Dropout randomly disables units during training but is normally turned off during inference. Batch normalization uses batch statistics during training and stored running statistics during inference. These differences are important because a model must be explicitly placed in the correct mode in many deep learning frameworks. For the inference stage itself, see “What is inference in deep learning?” . In short, training teaches the model by updating its weights, while inference uses the learned weights to make predictions without training them further.
What is a multilayer perceptron (MLP)?
A multilayer perceptron (MLP) is a feedforward neural network made of an input layer, one or more hidden layers, and an output layer. Each neuron in one layer is usually connected to every neuron in the next layer, which is why MLPs are also commonly described as fully connected neural networks . Basic structure A simple MLP looks like: Input layer → Hidden layer(s) → Output layer Each layer performs a weighted transformation followed by an activation function. A neuron typically computes: z = Wx + b and then applies an activation: a = f(z) Why are hidden layers important? Hidden layers allow the network to learn increasingly complex relationships between input features. With nonlinear activation functions, an MLP can model patterns that a single linear transformation cannot represent. How does an MLP learn? During training: Inputs move forward through the network. The model produces a prediction. A loss function measures the prediction error. Backpropagation calculates gradients. An optimizer updates the weights and biases. This process is repeated over many training examples. What are MLPs used for? MLPs can be used for tasks such as: Classification Regression Learning from tabular data Combining learned features before a final prediction They are less specialized than architectures such as CNNs for images or RNNs for sequential data, but they remain one of the fundamental neural network architectures. For the broader concept, see “What is a feedforward neural network?” and “What is a fully connected layer?” . In short, an MLP is a fully connected feedforward neural network with one or more hidden layers that learns nonlinear relationships between inputs and outputs.
How does a multilayer perceptron work?
A multilayer perceptron (MLP) works by passing input data forward through a series of fully connected layers, transforming the data at each layer until the network produces an output. 1. Input enters the network The input layer receives a vector of features. For example: [x1, x2, x3] These values are passed to the first hidden layer. 2. Each neuron computes a weighted sum A neuron combines its inputs using weights and a bias: z = w1x1 + w2x2 + ... + b The weights determine how strongly each input influences the neuron. 3. An activation function is applied The weighted sum is passed through a nonlinear activation function such as ReLU: a = f(z) This nonlinearity allows the MLP to learn complex relationships instead of behaving like one large linear transformation. 4. Hidden layers repeat the process The activations from one layer become the inputs to the next: Input → Hidden layer 1 → Hidden layer 2 → ... → Output layer Earlier layers can learn simpler combinations of features, while deeper layers can build more complex representations. 5. The output layer produces a prediction The final layer depends on the task. For example: Regression may use a single continuous output. Binary classification may use one probability like output. Multiclass classification may produce one score or probability for each class. 6. The network learns from its error During training, the prediction is compared with the correct target using a loss function. Backpropagation calculates how each weight contributed to the error, and an optimizer updates the weights and biases. Forward pass → Loss → Backpropagation → Weight update Repeating this process over many training examples gradually improves the model. For the architecture itself, see “What is a multilayer perceptron (MLP)?” . In short, an MLP repeatedly applies weighted transformations and nonlinear activation functions across multiple layers, then learns better weights through backpropagation and optimization.
What is a feedforward neural network?
A feedforward neural network is a neural network in which information moves in one direction—from the input layer, through any hidden layers, to the output layer. There are no feedback loops or recurrent connections that send information back to earlier layers. Basic structure A typical feedforward network looks like: Input layer → Hidden layer(s) → Output layer Each layer receives values from the previous layer, transforms them, and passes the result forward. What happens inside each layer? A neuron usually computes a weighted sum: z = Wx + b and then applies an activation function: a = f(z) The output of one layer becomes the input to the next layer. Why is it called feedforward? It is called feedforward because data only moves forward through the network during the forward pass. Unlike an RNN, a standard feedforward network does not maintain a hidden state from previous inputs. What are feedforward neural networks used for? They are commonly used for: Classification Regression Tabular data Function approximation Final prediction layers inside larger neural networks How is an MLP related? A multilayer perceptron (MLP) is one of the most common types of feedforward neural network. It usually contains one or more fully connected hidden layers with nonlinear activation functions. For more detail, see “What is a multilayer perceptron (MLP)?” . In short, a feedforward neural network processes information from input to output in one direction without recurrent feedback connections.
How does a feedforward neural network work?
A feedforward neural network works by passing information from the input layer through one or more hidden layers to the output layer, without sending information backward through recurrent connections. 1. Input values enter the network The network receives a set of input features: [x1, x2, x3, ...] These values are passed to the first hidden layer. 2. Neurons calculate weighted sums Each neuron combines its inputs using learned weights and a bias: z = w1x1 + w2x2 + ... + b The weights determine how strongly each input influences that neuron. 3. Activation functions add nonlinearity The weighted sum is passed through an activation function: a = f(z) Nonlinear activation functions allow the network to learn relationships that cannot be represented by a single linear transformation. 4. Information moves through hidden layers The output of one layer becomes the input to the next: Input → Hidden layer 1 → Hidden layer 2 → ... → Output Each layer transforms the representation before passing it forward. 5. The output layer produces the result The output layer is designed for the task. For example: Regression can produce a continuous value. Binary classification can produce a probability like score. Multiclass classification can produce scores or probabilities for several classes. 6. Training adjusts the weights During training, the network's prediction is compared with the correct target using a loss function. Backpropagation calculates gradients, and an optimizer updates the weights and biases. Forward pass → Loss → Backpropagation → Weight update The term feedforward describes the network architecture and forward computation. Backpropagation is still used during training to calculate how the parameters should change. For the basic definition, see “What is a feedforward neural network?” . In short, a feedforward neural network transforms inputs layer by layer in one forward direction, then learns better weights by minimizing prediction error during training.
What is the difference between shallow and deep neural networks?
The main difference between shallow and deep neural networks is the number of hidden layers used to transform the input before producing an output. A shallow network has only a small number of hidden layers, while a deep neural network has many. Shallow neural network A shallow neural network typically has: Input layer → One or a few hidden layers → Output layer It can learn useful nonlinear relationships, but its ability to build complex hierarchical representations is more limited. Deep neural network A deep neural network contains multiple hidden layers: Input → Hidden 1 → Hidden 2 → Hidden 3 → ... → Output Each layer can transform the representation learned by the previous layer. This allows deeper networks to learn features in stages. For example, in image recognition: Pixels → Edges → Shapes → Object parts → Object Why can deeper networks be more powerful? Depth allows a network to represent some complex functions more efficiently by composing many simpler transformations. Deep networks are especially useful for complex data such as: Images Audio Text Video Is a deeper network always better? No. Adding more layers can also make training harder and may increase: Computational cost Memory usage Training time Risk of overfitting Optimization problems such as vanishing or exploding gradients Techniques such as normalization, better initialization, and residual connections help make very deep networks easier to train. There is no universal number of layers that officially separates a shallow network from a deep one; the terms are relative. For the basic concept of network depth, see “What does the depth of a neural network mean?” . In short, shallow networks use relatively few hidden layers, while deep networks use many layers to learn increasingly complex hierarchical representations.
Why do neural networks need hidden layers?
Neural networks need hidden layers because hidden layers allow the network to transform raw inputs into increasingly useful internal representations. Without hidden layers, a neural network is essentially limited to learning a direct relationship between the input and output. What happens in a hidden layer? Each hidden layer receives values from the previous layer, applies learned weights and biases, and then usually applies a nonlinear activation function. A simplified flow is: Input → Hidden layer → Hidden layer → Output Each hidden layer can learn a different transformation of the data. Why are hidden layers important? Hidden layers help neural networks learn: Nonlinear relationships Interactions between features Hierarchical patterns More useful representations for the final prediction Example In an image recognition network, different layers may gradually learn: Pixels → Edges → Textures → Shapes → Object features The network does not need these intermediate features to be manually programmed. It learns them from data during training. Why is nonlinearity important? If hidden layers only performed linear transformations, stacking many of them would still behave like a single linear transformation. Nonlinear activation functions allow hidden layers to represent much more complex functions. Do more hidden layers always help? No. Additional hidden layers increase the model's capacity, but they can also increase computation, training difficulty, and the risk of overfitting. The appropriate number of hidden layers depends on the complexity of the task and the available data. For the broader architectural idea, see “What are input, hidden, and output layers in a neural network?” . In short, hidden layers give neural networks the ability to learn intermediate representations and complex nonlinear relationships between inputs and outputs.
What happens inside a hidden layer of a neural network?
Inside a hidden layer, each neuron receives values from the previous layer, combines them using weights and a bias, and then usually applies an activation function. This transforms the incoming representation into a new one that can be more useful for the task. Step 1: Receive inputs A hidden neuron receives activations from the previous layer: x1, x2, x3, ... Step 2: Compute a weighted sum The neuron calculates: z = w1x1 + w2x2 + ... + b This value is called the neuron's pre activation value . Step 3: Apply an activation function The pre activation is passed through an activation function: a = f(z) For example, ReLU may convert negative values to zero while keeping positive values. Step 4: Pass the result forward The activation becomes input to neurons in the next layer. A hidden layer therefore performs a transformation like: Previous representation → Weighted transformation → Activation → New representation What does the hidden layer learn? During training, the network adjusts the weights and biases so hidden neurons respond to useful patterns in the data. Different layers can learn different levels of representation. For example, in image recognition: Pixels → Edges → Textures → Shapes → Object features The exact meaning of a hidden neuron's output is learned from data rather than manually programmed. For the broader purpose of hidden layers, see “Why do neural networks need hidden layers?” . In short, a hidden layer takes the previous layer's outputs, applies learned weighted transformations and nonlinear activation functions, and produces a new representation for the next layer.
How many hidden layers should a neural network have?
There is no single correct number of hidden layers for every neural network. The right depth depends on the complexity of the problem, the amount of data available, and how difficult the model is to train. A simple rule of thumb For relatively simple problems, one or two hidden layers may be enough. For complex tasks such as image recognition, speech processing, or language modeling, networks may use many hidden layers. Why use more hidden layers? Additional hidden layers allow the network to build more complex hierarchical representations. For example: Pixels → Edges → Shapes → Object parts → Object Each layer can transform the representation produced by the previous layer. Why not keep adding layers? More layers also increase: Computational cost Memory usage Training time Risk of overfitting Optimization difficulty Very deep networks can also suffer from problems such as vanishing gradients or degradation if the architecture is not designed carefully. How should the number of layers be chosen? A practical approach is to: Start with a relatively simple architecture. Measure performance on validation data. Increase depth only when the model is underfitting or the task clearly benefits from deeper representations. Use techniques such as normalization and residual connections when training very deep networks. The optimal depth is therefore usually found through experimentation rather than a fixed formula. For the meaning of network depth itself, see “What does the depth of a neural network mean?” . In short, a neural network should have enough hidden layers to model the task well, but not so many that unnecessary complexity makes training slower or less reliable.