Backpropagation Step by Step - [The Mathematical Approach]
While studying backpropagation, I came across this great mathematical explanation or breakdown of how it goes. Matt Mazur, a data scientist working for Help Scout, wrote about it on his personal blog.
I am so excited about the mathematical details that power neural networks that I enjoy writing down on a piece of paper how the actual computations work, both for forward propagation (the logits and the activations) and for backpropagation with gradient descent (the partial derivatives).
This passion and excitement rose inside of me as doing the courses in the Deep Learning specializations of Professor Andrew Ng at Coursera. This part of neural networks is greatly explain in the first course.
Anyway, backpropagation simply means back propagating the error you get as you reach the output of a network, with the purpose of updating the weights to minimize this error. Matt details both the forward pass and the backward pass through a neural network, with the equations and derivatives that they imply.
So, if you're a geek that's excited about this stuff, I'd recommend taking a piece of paper and replicating the work done by Matt on his blog. Also, do check the visualizations he provides as they enhance the understanding of backprop.
To stay in touch with me, follow @cristi
Cristi Vlad Self-Experimenter and Author
I always use chain rule of differentiation to understand backpropagation. Since neural network can be represented as f(g(h(...(input)...))) where each function is a layer of the neural network, while trying to minimize the error function which is another function of (predicted-actual), we differentiate this function of function representation, resulting in usage of chain rule.
@cristi can you help me out in derivatives and integration i will be happy ?
Sure, can you be more specific?
Very interesting @cristi, as a mathematician Neural Networks and Genetical Algorithms are one my top list of Topics.
You got yourself a new follower :) Saludos!
I new follower. Thanks
You got yourself one too!!!
Super. Very nice
Thanks for the link, looks very interesting.
great information the one of your publication, generally I practice at every moment the mathematics since my university career is based on pure calculations. Greetings cristi
I always take very good with math since I was in the school of Christ. regards