# Artificial Neural Networks : Backpropagation

After forward propagation, we backpropagate the error gradient (i.e. derivative of cost function or error function with respect to weights and biases) to update weight and bias parameters in order to reduce the error.

To understand the mathematics behind the backpropagation, Let's work on an example. For that let's take the predicted outputs of data obtained from forward propagation.

Since it is a binary classification problem, cost function or error function of the network is given by,

As we need the error gradient to update parameters, we have to find the derivative of error function with respect to weights and bias.

Let's derive the derivative of E with respect to any weight Wj,

Then,

Then,

From that we can write the derivative of E with respect to weights W as,

Likewise we can find the derivative of E with respect to bias b.

Using the derived equations we can find our error gradients. Error gradients with respect to weights are,

Error gradient with respect to bias is,

After finding the error gradients we can update our weights and bias using the following equations,

If we assign our learning rate as 0.1, the our updated weights and bias are,

This how using backpropagation, weights and bias are updated. Then the network will perform the forward propagation using the updated weights and bias and predict the outputs. From the predicted outputs, using backpropagation, weights and bias will be updated again and the process of forward propagation and backpropagation will continue until the cost function of the network reaches the minimum. This how the network learns from the training samples.

In the next article we will discuss the implementation of forward propagation and backpropagation from scratch and using machine learning libraries.

## Related articles

### How does 'A Content Based Recommender System' work?

This article mainly focuses on the theory behind Content Based Recommender Systems and explains it using an Example.

### Visualizing the Output Images of the Convolutional Layers of a CNN

Have you ever wondered ‘How do the output images of the convolutional layers of a Convolutional Neural Network (CNN) look like?’. If yes, then this article is for you.

### Implementation of ANN for MNIST Handwritten Digits Classification

This article explains the basics of how to build an ANN model for the classification of MNIST dataset (Step by Step).