What is function learning?

My batch gradient descent algorithm is not converging (i.e., the cost function does not decrease with every iteration, even though I have set my learning rate quiet small and have verified with different learning rates, both large and even smaller). What could be possibly be going wrong?

  • Answer:

    The most likely answer is a bug in your implementation. :) Your learning rate may not actually be as small as you think, or not small enough for this data set. You may have a normalization problem, where your data is "small" relative to learning rate only along a few dimensions.

Sean Owen at Quora Visit the source

Was this solution helpful to you?

Just Added Q & A:

Find solution

For every problem there is a solution! Proved by Solucija.

  • Got an issue and looking for advice?

  • Ask Solucija to search every corner of the Web for help.

  • Get workable solutions and helpful tips in a moment.

Just ask Solucija about an issue you face and immediately get a list of ready solutions, answers and tips from other Internet users. We always provide the most suitable and complete answer to your question at the top, along with a few good alternatives below.