# Non-Convex Loss Function

I am trying to understand gradient descent algorithm by plotting the error vs value of parameters in the function. What would be an example of a simple function of the form y = f(x) with just just one input variable x and two parameters w1 and w2 such that it has a non-convex(has multiple minima) loss function ? What i am trying to achieve is something similar to this : How does one know if the function has a non-convex loss function without plotting the graph ?

• It seems (following your latest edit) you strongly associate non-convexity with the presence of multiple local minima; this is not a very coherent idea. I would suggest you keep the question in its original form (asking about convexity) and ask a second question directly on the matter of multiple local minima. – usεr11852 May 13 '17 at 17:48
• – melwin_jose May 13 '17 at 18:05
• Saw it. Sorted. – usεr11852 May 13 '17 at 18:24

• Sorry I missed that! Simplest form I can think of is: $\log(x)$ as a non-convex loss function example. The 2nd derivative of it is $\frac{-1}{x^2}$. So say $f(x)=\log(w_1x+w_2)$. Then $f''(x)= \frac{-w_1^2}{(w_1x+w_2)^2}$. – usεr11852 May 13 '17 at 14:56
• What is shown in the plot you attached is a non-convex function, $\log(x)$ is a concave function (or convex-upwards if you wish). From an optimisation stand-point you can always turn a concave function $f(x)$ to concave by simply optimising $-f(x)$. The problem when optimising a loss function relates to the presence of (potentially multiple) local minima and flatness. Curvature information allows us to prove certain vital properties of $f(x)$. – usεr11852 May 13 '17 at 17:23
• No probs. I suspected that there was some misunderstanding involved... :D The idea is the same though. Using calculus, multiple local minima would relate to multiple instances of first derivatives being zero and second derivatives being positive. Take $x \sin(w1x+w2)$ for example; it has infinite on them. – usεr11852 May 13 '17 at 17:55