[Quora] What is the difference between evolutionary algorithms and deep learning?

You can’t really compare those as while a deep neural network is a data structure of sorts, an evolutionary algorithm is a way of learning.

This means you could have a model that is a deep learning evolutionary algorithm (the English sort of break down when you say it like that). However, the concepts are compatible.

To illustrate this, let’s start with something simple; a neural network.

Above there’s a general model of a simple perceptron (a special type of neural network that doesn’t have any hidden layers (is not deep). This one also only has one output value, Z.

So how does a deep neural network differ?

I actually already mentioned that. A deep neural network has multiple hidden layers with learnable weights.

One such example is the one below:

Okay, so now that we know what a deep neural network is, let’s examine how evolutionary algorithms work.

Evolutionary algorithms when applied to neural networks is a way of updating the weights of network.

Let’s remind ourselves of how the weights are usually updated; that is using gradient descent backpropagation. The backpropagation ensures that the weights are updated along the gradient to ensure fast learning.

Evolutionary learning, on the other hand, would not necessarily follow the gradient, but instead it just randomly, and blindly changes the weights hoping that it will increase the accuracy.

In general, evolutionary algorithms work by randomly changing parameters, and if it’s an improvement it saves the new “genetic code”. One nice thing about this, is that it can be done in parallel, and the actors can broadcast their genetic code for horizontal adaptation. However, the aimless nature of the “evolution” is a very inefficient way of learning.

This also means that almost always gradient descent is better than evolutionary learning; simply because it’s better than literally random.

And this is why evolutionary algorithms have become quite unpopular recently.