# Wrong fitted parameters in multivariate linear regression?

I am implementing multivariate linear regression using numpy, pandas and matplotlib. I then compared my results to in-built scipy optimizing methods. It looks like the thetas which I am finding using gradient descent are different to those obtained using scipy.optimize.

I am reading data from a file which looks like this:

data.head()

ldr1  ldr2  servo
0   971   956     -2
1   691   825   -105
2   841   963    -26
3   970   731     44
4   755   939    -69 I proceed to implement gradient descent and computing the cost function. I include reading from file and plotting for completeness.

def read_data(file):
# read in data using pandas
data.columns = ["ldr1", "ldr2", "servo"]    # read the data
# print(file_data)
return data

def plot_data(file_data):
ldr1 = my_data.iloc[:, 0:1]
ldr2 = my_data.iloc[:, 1:2]
servo_correction = my_data.iloc[:, 2:3]

fig = plt.figure()
ax = Axes3D(fig)
ax.scatter(ldr2, ldr1, servo_correction)
ax.set_zlabel('Delta Servo')
plt.xlabel("LDR2")
plt.ylabel("LDR1")
plt.gca().invert_xaxis()
plt.show()
return ldr1, ldr2, servo_correction

# compute cost
def compute_cost(X, y, theta):
to_be_summed = np.power(((X @ theta.T)-y), 2)
return np.sum(to_be_summed)/(2 * len(X))

def gradient_descent(X, y, theta, iters, alpha):
cost = np.zeros(iters)
for i in range(iters):
theta = theta - (alpha / len(X)) * np.sum(X * (X @ theta.T - y), axis=0)
cost[i] = compute_cost(X, y, theta)
return theta, cost


I call these functions like so:

my_data = read_data(filename)
ldr1, ldr2, servo = plot_data(my_data)

# we need to normalize the features using mean normalization
my_data = (my_data - my_data.mean())/my_data.std()

# setting the matrices
X = my_data.iloc[:, 0:2]
ones = np.ones([X.shape, 1])
X = np.concatenate((ones, X), axis=1)

y = my_data.iloc[:, 2:3].values  # values converts it from pandas.core.frame.DataFrame to numpy.ndarray
theta = np.zeros([1, 3])

# set hyper parameters
alpha = 0.01
iterations = 1000

# running the gd and cost function
g, cost = gradient_descent(X, y, theta, iterations, alpha)
print("Thetas: ", g)

finalCost = compute_cost(X, y, g)
print("Final Cost: ", finalCost)


I am trying to fit the plane of best fit to this data. Currently my output is:

Thetas:  [[-3.86865143e-17  8.47885685e-01 -5.39083511e-01]]
Final Cost:  0.11972883176814067 I then used scipy.optimize.curve_fit and got different values for fittedParameters:

if __name__ == "__main__":
data = [xData, yData, zData]

# here a non-linear surface fit is made with scipy's curve_fit()
fittedParameters, pcov = scipy.optimize.curve_fit(func, [xData, yData], zData, p0 = initialParameters)
print(fittedParameters)

SurfacePlot(func, data, fittedParameters)

fitted prameters [   0.26654135   -0.15218007 -107.79915373] Any suggestions on what I am doing wrong?

Data set can be accessed here: https://www.dropbox.com/s/wycoi7gm2sbjr95/

EDIT

I found the issue, I am plotting thetas I got from gradient descent on normalized data on top of the original 'un-normalized' data. Trying to figure out how to get thetas for original data set in order to be able to visualise the plane of best fit.