1
$\begingroup$

I am implementing multivariate linear regression using numpy, pandas and matplotlib. I then compared my results to in-built scipy optimizing methods. It looks like the thetas which I am finding using gradient descent are different to those obtained using scipy.optimize.

I am reading data from a file which looks like this:

data.head()

   ldr1  ldr2  servo
0   971   956     -2
1   691   825   -105
2   841   963    -26
3   970   731     44
4   755   939    -69

enter image description here

I proceed to implement gradient descent and computing the cost function. I include reading from file and plotting for completeness.

def read_data(file):
    # read in data using pandas
    data = pd.read_csv(file, sep=" ", header=None)
    data.columns = ["ldr1", "ldr2", "servo"]    # read the data
    print(data.head())
    # print(file_data)
    return data


def plot_data(file_data):
    ldr1 = my_data.iloc[:, 0:1]
    ldr2 = my_data.iloc[:, 1:2]
    servo_correction = my_data.iloc[:, 2:3]

    fig = plt.figure()
    ax = Axes3D(fig)
    ax.scatter(ldr2, ldr1, servo_correction)
    ax.set_zlabel('Delta Servo')
    plt.xlabel("LDR2")
    plt.ylabel("LDR1")
    plt.gca().invert_xaxis()
    plt.show()
    return ldr1, ldr2, servo_correction


# compute cost
def compute_cost(X, y, theta):
    to_be_summed = np.power(((X @ theta.T)-y), 2)
    return np.sum(to_be_summed)/(2 * len(X))


# gradient descent
def gradient_descent(X, y, theta, iters, alpha):
    cost = np.zeros(iters)
    for i in range(iters):
        theta = theta - (alpha / len(X)) * np.sum(X * (X @ theta.T - y), axis=0)
        cost[i] = compute_cost(X, y, theta)
    return theta, cost

I call these functions like so:

my_data = read_data(filename)
ldr1, ldr2, servo = plot_data(my_data)

# we need to normalize the features using mean normalization
my_data = (my_data - my_data.mean())/my_data.std()
# print(my_data.head())

# setting the matrices
X = my_data.iloc[:, 0:2]
ones = np.ones([X.shape[0], 1])
X = np.concatenate((ones, X), axis=1)

y = my_data.iloc[:, 2:3].values  # values converts it from pandas.core.frame.DataFrame to numpy.ndarray
theta = np.zeros([1, 3])

# set hyper parameters
alpha = 0.01
iterations = 1000

# running the gd and cost function
g, cost = gradient_descent(X, y, theta, iterations, alpha)
print("Thetas: ", g)

finalCost = compute_cost(X, y, g)
print("Final Cost: ", finalCost)

I am trying to fit the plane of best fit to this data. Currently my output is:

Thetas:  [[-3.86865143e-17  8.47885685e-01 -5.39083511e-01]]
Final Cost:  0.11972883176814067

enter image description here

I then used scipy.optimize.curve_fit and got different values for fittedParameters:

if __name__ == "__main__":
    data = [xData, yData, zData]

    # here a non-linear surface fit is made with scipy's curve_fit()
    fittedParameters, pcov = scipy.optimize.curve_fit(func, [xData, yData], zData, p0 = initialParameters)
    print(fittedParameters)

    SurfacePlot(func, data, fittedParameters)

fitted prameters [   0.26654135   -0.15218007 -107.79915373]

enter image description here

Any suggestions on what I am doing wrong?

Data set can be accessed here: https://www.dropbox.com/s/wycoi7gm2sbjr95/


EDIT

I found the issue, I am plotting thetas I got from gradient descent on normalized data on top of the original 'un-normalized' data. Trying to figure out how to get thetas for original data set in order to be able to visualise the plane of best fit.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.