I am implementing multivariate linear regression using numpy, pandas and matplotlib. I then compared my results to in-built scipy optimizing methods. It looks like the thetas which I am finding using gradient descent are different to those obtained using scipy.optimize.

I am reading data from a file which looks like this:


   ldr1  ldr2  servo
0   971   956     -2
1   691   825   -105
2   841   963    -26
3   970   731     44
4   755   939    -69

enter image description here

I proceed to implement gradient descent and computing the cost function. I include reading from file and plotting for completeness.

def read_data(file):
    # read in data using pandas
    data = pd.read_csv(file, sep=" ", header=None)
    data.columns = ["ldr1", "ldr2", "servo"]    # read the data
    # print(file_data)
    return data

def plot_data(file_data):
    ldr1 = my_data.iloc[:, 0:1]
    ldr2 = my_data.iloc[:, 1:2]
    servo_correction = my_data.iloc[:, 2:3]

    fig = plt.figure()
    ax = Axes3D(fig)
    ax.scatter(ldr2, ldr1, servo_correction)
    ax.set_zlabel('Delta Servo')
    return ldr1, ldr2, servo_correction

# compute cost
def compute_cost(X, y, theta):
    to_be_summed = np.power(((X @ theta.T)-y), 2)
    return np.sum(to_be_summed)/(2 * len(X))

# gradient descent
def gradient_descent(X, y, theta, iters, alpha):
    cost = np.zeros(iters)
    for i in range(iters):
        theta = theta - (alpha / len(X)) * np.sum(X * (X @ theta.T - y), axis=0)
        cost[i] = compute_cost(X, y, theta)
    return theta, cost

I call these functions like so:

my_data = read_data(filename)
ldr1, ldr2, servo = plot_data(my_data)

# we need to normalize the features using mean normalization
my_data = (my_data - my_data.mean())/my_data.std()
# print(my_data.head())

# setting the matrices
X = my_data.iloc[:, 0:2]
ones = np.ones([X.shape[0], 1])
X = np.concatenate((ones, X), axis=1)

y = my_data.iloc[:, 2:3].values  # values converts it from pandas.core.frame.DataFrame to numpy.ndarray
theta = np.zeros([1, 3])

# set hyper parameters
alpha = 0.01
iterations = 1000

# running the gd and cost function
g, cost = gradient_descent(X, y, theta, iterations, alpha)
print("Thetas: ", g)

finalCost = compute_cost(X, y, g)
print("Final Cost: ", finalCost)

I am trying to fit the plane of best fit to this data. Currently my output is:

Thetas:  [[-3.86865143e-17  8.47885685e-01 -5.39083511e-01]]
Final Cost:  0.11972883176814067

enter image description here

I then used scipy.optimize.curve_fit and got different values for fittedParameters:

if __name__ == "__main__":
    data = [xData, yData, zData]

    # here a non-linear surface fit is made with scipy's curve_fit()
    fittedParameters, pcov = scipy.optimize.curve_fit(func, [xData, yData], zData, p0 = initialParameters)

    SurfacePlot(func, data, fittedParameters)

fitted prameters [   0.26654135   -0.15218007 -107.79915373]

enter image description here

Any suggestions on what I am doing wrong?

Data set can be accessed here: https://www.dropbox.com/s/wycoi7gm2sbjr95/


I found the issue, I am plotting thetas I got from gradient descent on normalized data on top of the original 'un-normalized' data. Trying to figure out how to get thetas for original data set in order to be able to visualise the plane of best fit.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.