# Multiple linear regression with very large cost

So, I'm trying to enter the Data Science world but struggling with a very simpel exercise. I'm using a dataset to get personal medical costs from a bunch of personal info. The database look like this:

Right now, this is my code:

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt

sex = {'female': 0, 'male': 1}
smoker = {'no': 0, 'yes': 1}
region = {'southwest': 0, 'southeast': 1, 'northwest': 2, 'northeast': 3}

def filterData(data):
data['smoker'] = [smoker[item] for item in data['smoker']]
data['sex'] = [sex[item] for item in data['sex']]
data['region'] = [region[item] for item in data['region']]
data['charges'] = [round(item, 2) for item in data['charges']]

return data

data = filterData(data)

x = data[['age', 'sex', 'bmi', 'children', 'smoker']]
x.insert(0, 'coefficient', np.ones(len(data.index)))
y = data['charges']

theta = np.zeros(len(x.columns))

alpha = 0.00003
iters = 1000

def getMSE(x, y, theta):
predY = np.sum(x * theta, 1)
meanSquare = np.power((predY - y), 2)
MSE = np.sum(meanSquare) / (2 * len(x))

return MSE

def gradientDescent(x, y, theta, alpha, iters):
cost = []

for iteration in range(iters):
predY = np.sum(x * theta, 1)
loss = predY - y
for j in range(len(theta)):
for m in range(len(x)):

cost.append(getMSE(x, y, theta))

return theta, cost

x = np.array(x)
y = np.array(y)
theta = np.array(theta).T

MSE = getMSE(x, y, theta)
theta, cost = gradientDescent(x, y, theta, alpha, iters)

print(theta)
plt.plot(list(range(iters)), cost, '-r')
plt.show()

I think that some of the methods are not working properly, because even after the model training, my cost curve looks like this:

I tried to change the iters and learning rate but nothing seems to make it better

• Some of these values for # charges are huge, as is the spread. Consider what happens if you normalize your data before running regression on it. It'll make the loss more understandable. The shape of the curve should wind up the same, but the y-axis will certainly be smaller. Commented Apr 20, 2021 at 3:03
• Hm, ok, I understand. But the methods are right? Commented Apr 20, 2021 at 3:06
• Your gradient descent function has problems. Just from skimming it, the first two I noticed are (1) Why is your loss $\hat{y} - y$ instead of $(\hat{y} - y)^2$? (2) You need to reset gradient more often, or you update $\theta_2$ based on $\theta_1$, you update $\theta_3$ based on $\theta_1$ and $\theta_2$, etc. Commented Apr 20, 2021 at 3:09
• (1) The loss function in referent to the derivative of each theta term, in this case it's not predY - y? (2) For gradient, I should reset every j in theta loop? Commented Apr 20, 2021 at 16:07

$$\frac{x - \mu}{\sigma}$$
for both the data columns and the labels. After you make predictions, you can then bring the data back to the original scale by storing the normalization values and doing $$\hat{y}\sigma_y + \mu_y$$