For a school project, we have to implement a linear regression with gradient descent. The data is the price of cars given the mileage, so at the end I should obtain a negative slope when I visualise the result.
I tried to calculate the theta with the normal equation and it works like a charm :
But when I use the gradient descent the theta change quickly for a positive slope that cut the data perpendicular to the expected slope then the thetas update become extremely slow :
For the gradient descent I use a very small alpha (1e-10!) and I can't increase it, otherwise it failed by overshooting the cost function.
My problem is, I don't understand if (1) the problem is not well suited for gradient descent, (2) the data values are too big to converge quickly to the good solution or (3) another problem I don't think of.
Here is the python code of my function, I use numpy array as container:
def gradDescentIteration(X, y, theta, alpha, numIter): Xones = fll.append_bias(X).T # Append a column of 1 for the bias coefficient m = np.shape(y) for i in xrange(numIter): H = np.dot(theta, Xones) diff = H - y.T diffm = np.dot(Xones, diff.T).T sigma = diffm * 1 / float(m) theta = theta - alpha * sigma return theta
Thanks for reading
Edit1 : dead link images