So, I'm trying to enter the Data Science world but struggling with a very simpel exercise. I'm using a dataset to get personal medical costs from a bunch of personal info. The database look like this:
Right now, this is my code:
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
data = pd.read_csv('./data.csv')
sex = {'female': 0, 'male': 1}
smoker = {'no': 0, 'yes': 1}
region = {'southwest': 0, 'southeast': 1, 'northwest': 2, 'northeast': 3}
def filterData(data):
data['smoker'] = [smoker[item] for item in data['smoker']]
data['sex'] = [sex[item] for item in data['sex']]
data['region'] = [region[item] for item in data['region']]
data['charges'] = [round(item, 2) for item in data['charges']]
return data
data = filterData(data)
x = data[['age', 'sex', 'bmi', 'children', 'smoker']]
x.insert(0, 'coefficient', np.ones(len(data.index)))
y = data['charges']
theta = np.zeros(len(x.columns))
alpha = 0.00003
iters = 1000
def getMSE(x, y, theta):
predY = np.sum(x * theta, 1)
meanSquare = np.power((predY - y), 2)
MSE = np.sum(meanSquare) / (2 * len(x))
return MSE
def gradientDescent(x, y, theta, alpha, iters):
cost = []
for iteration in range(iters):
predY = np.sum(x * theta, 1)
loss = predY - y
gradient = 0
for j in range(len(theta)):
for m in range(len(x)):
gradient += loss[m] - x[m][j]
theta[j] -= (alpha/len(x)) * gradient
cost.append(getMSE(x, y, theta))
return theta, cost
x = np.array(x)
y = np.array(y)
theta = np.array(theta).T
MSE = getMSE(x, y, theta)
theta, cost = gradientDescent(x, y, theta, alpha, iters)
print(theta)
plt.plot(list(range(iters)), cost, '-r')
plt.show()
I think that some of the methods are not working properly, because even after the model training, my cost curve looks like this:
I tried to change the iters and learning rate but nothing seems to make it better
# charges
are huge, as is the spread. Consider what happens if you normalize your data before running regression on it. It'll make the loss more understandable. The shape of the curve should wind up the same, but the y-axis will certainly be smaller. $\endgroup$gradient
more often, or you update $\theta_2$ based on $\theta_1$, you update $\theta_3$ based on $\theta_1$ and $\theta_2$, etc. $\endgroup$gradient
, I should reset everyj
in theta loop? $\endgroup$