Multiple linear regression with very large cost

Question

So, I'm trying to enter the Data Science world but struggling with a very simpel exercise. I'm using a dataset to get personal medical costs from a bunch of personal info. The database look like this:

Right now, this is my code:

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt

data = pd.read_csv('./data.csv')

sex = {'female': 0, 'male': 1}
smoker = {'no': 0, 'yes': 1}
region = {'southwest': 0, 'southeast': 1, 'northwest': 2, 'northeast': 3}


def filterData(data):
    data['smoker'] = [smoker[item] for item in data['smoker']]
    data['sex'] = [sex[item] for item in data['sex']]
    data['region'] = [region[item] for item in data['region']]
    data['charges'] = [round(item, 2) for item in data['charges']]

    return data


data = filterData(data)

x = data[['age', 'sex', 'bmi', 'children', 'smoker']]
x.insert(0, 'coefficient', np.ones(len(data.index)))
y = data['charges']

theta = np.zeros(len(x.columns))

alpha = 0.00003
iters = 1000


def getMSE(x, y, theta):
    predY = np.sum(x * theta, 1)
    meanSquare = np.power((predY - y), 2)
    MSE = np.sum(meanSquare) / (2 * len(x))

    return MSE


def gradientDescent(x, y, theta, alpha, iters):
    cost = []

    for iteration in range(iters):
        predY = np.sum(x * theta, 1)
        loss = predY - y
        gradient = 0
        for j in range(len(theta)):
            for m in range(len(x)):
                gradient += loss[m] - x[m][j]
            theta[j] -= (alpha/len(x)) * gradient

        cost.append(getMSE(x, y, theta))

    return theta, cost


x = np.array(x)
y = np.array(y)
theta = np.array(theta).T

MSE = getMSE(x, y, theta)
theta, cost = gradientDescent(x, y, theta, alpha, iters)

print(theta)
plt.plot(list(range(iters)), cost, '-r')
plt.show()

I think that some of the methods are not working properly, because even after the model training, my cost curve looks like this:

I tried to change the iters and learning rate but nothing seems to make it better

Answer 1

The shape of your curve looks correct. The y values in your data are pretty large and you have done no normalization. This would probably work much better if you first center the data by

$$ \frac{x - \mu}{\sigma} $$

for both the data columns and the labels. After you make predictions, you can then bring the data back to the original scale by storing the normalization values and doing $\hat{y}\sigma_y + \mu_y$