Gradient descent algorithm is one of the most popuarl algorithms for finding optimal parameters for most machine learning models including neural networks. The basic method that this algorithm uses is to find optimal values for the parameters that define your ‘cost function’.