0
$\begingroup$

So, I'm trying to enter the Data Science world but struggling with a very simpel exercise. I'm using a dataset to get personal medical costs from a bunch of personal info. The database look like this:

enter image description here

Right now, this is my code:

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt

data = pd.read_csv('./data.csv')

sex = {'female': 0, 'male': 1}
smoker = {'no': 0, 'yes': 1}
region = {'southwest': 0, 'southeast': 1, 'northwest': 2, 'northeast': 3}


def filterData(data):
    data['smoker'] = [smoker[item] for item in data['smoker']]
    data['sex'] = [sex[item] for item in data['sex']]
    data['region'] = [region[item] for item in data['region']]
    data['charges'] = [round(item, 2) for item in data['charges']]

    return data


data = filterData(data)

x = data[['age', 'sex', 'bmi', 'children', 'smoker']]
x.insert(0, 'coefficient', np.ones(len(data.index)))
y = data['charges']

theta = np.zeros(len(x.columns))

alpha = 0.00003
iters = 1000


def getMSE(x, y, theta):
    predY = np.sum(x * theta, 1)
    meanSquare = np.power((predY - y), 2)
    MSE = np.sum(meanSquare) / (2 * len(x))

    return MSE


def gradientDescent(x, y, theta, alpha, iters):
    cost = []

    for iteration in range(iters):
        predY = np.sum(x * theta, 1)
        loss = predY - y
        gradient = 0
        for j in range(len(theta)):
            for m in range(len(x)):
                gradient += loss[m] - x[m][j]
            theta[j] -= (alpha/len(x)) * gradient

        cost.append(getMSE(x, y, theta))

    return theta, cost


x = np.array(x)
y = np.array(y)
theta = np.array(theta).T

MSE = getMSE(x, y, theta)
theta, cost = gradientDescent(x, y, theta, alpha, iters)

print(theta)
plt.plot(list(range(iters)), cost, '-r')
plt.show()

I think that some of the methods are not working properly, because even after the model training, my cost curve looks like this:

enter image description here

I tried to change the iters and learning rate but nothing seems to make it better

$\endgroup$
4
  • $\begingroup$ Some of these values for # charges are huge, as is the spread. Consider what happens if you normalize your data before running regression on it. It'll make the loss more understandable. The shape of the curve should wind up the same, but the y-axis will certainly be smaller. $\endgroup$ Apr 20, 2021 at 3:03
  • $\begingroup$ Hm, ok, I understand. But the methods are right? $\endgroup$ Apr 20, 2021 at 3:06
  • $\begingroup$ Your gradient descent function has problems. Just from skimming it, the first two I noticed are (1) Why is your loss $\hat{y} - y$ instead of $(\hat{y} - y)^2$? (2) You need to reset gradient more often, or you update $\theta_2$ based on $\theta_1$, you update $\theta_3$ based on $\theta_1$ and $\theta_2$, etc. $\endgroup$ Apr 20, 2021 at 3:09
  • $\begingroup$ (1) The loss function in referent to the derivative of each theta term, in this case it's not predY - y? (2) For gradient, I should reset every j in theta loop? $\endgroup$ Apr 20, 2021 at 16:07

1 Answer 1

0
$\begingroup$

The shape of your curve looks correct. The y values in your data are pretty large and you have done no normalization. This would probably work much better if you first center the data by

$$ \frac{x - \mu}{\sigma} $$

for both the data columns and the labels. After you make predictions, you can then bring the data back to the original scale by storing the normalization values and doing $\hat{y}\sigma_y + \mu_y$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.