1
$\begingroup$

I'm hoping to better understand the predicted output of the dependent variable using a multiple linear regression. Specifically, I'm getting negative predicted outputs when altering specific independent variables.

I'll display an example df below and explain the independent variables

import pandas as pd
import numpy as np
from sklearn.linear_model import LinearRegression
from sklearn.cross_validation import train_test_split
from statsmodels.stats.outliers_influence import variance_inflation_factor
from statsmodels.tools.tools import add_constant

d = ({
    'Date' :    ['01/01/18','01/01/18','01/01/18','01/01/18','02/01/18','02/01/18','02/01/18','02/01/18'],
    'Country' :    ['US','US','US','MX','US','US','MX','MX'],   
    'Occurrences' :    [1,5,3,4,2,5,10,2],   
    'Turnover' :    [100000,40000,500000,8000,10000,300000,80000,1000],     
    'Medium' :    ['S1','S2','S1','S2','S1','S1','S1','S2'],     
    'Day' :    ['Saturday','Saturday','Saturday','Saturday','Sunday','Sunday','Sunday','Sunday'],              
})

df = pd.DataFrame(data=d)

# One hot Encoding           
one_hot = pd.get_dummies(df[['Country','Medium','Day']], drop_first=True)
df = df.join(one_hot)

df = df[['Country_US','Medium_S2','Day_Sunday','Occurrences','Turnover']]

Model: 

# Extract features and labels
features = df.iloc[:,0:-1].values
labels = df.iloc[:,-1].values

# Cross validation
X_train, X_test, y_train, y_test = train_test_split(features, labels, test_size = 0.3, random_state = 0)

# Create Linear Model
regressor = LinearRegression()
regressor.fit(X_train, y_train)

# CoEfficient and Intercept
#print(regressor.coef_)
#print(regressor.intercept_)

# Predict the Output
y_Pred = regressor.predict(X_test)
print(regressor.predict([0, 1, 0, 1]))

# Determine multicollinearity
X = add_constant(df)

VIF = pd.Series([variance_inflation_factor(X.values, i) 
               for i in range(X.shape[1])], 
              index=X.columns)

print(VIF)

const          19.914550
Country_US      2.029087
Medium_S2       2.165254
Day_Sunday      1.495758
Occurrences     1.364778
Turnover        1.706089

So the variables I've plugged in are: Country_Group : 0 Medium_S2 : 1 Day_Sunday = 0 Occurrences : 1

I understand Linear regression does not respect the bounds of 0. But is there a method I can ensure the output is a positive number without affecting the validity of the model?

The minimum output that can be generated is zero. It cannot be a negative number.

Output:

[-95500.]
Improve this question
$\endgroup$
4
  • $\begingroup$ What is your research question? Are Occurrences categorical or continuous? Is there a reason why Time Started would need to be continuous? $\endgroup$
    – Jay Schyler Raadt
    Jun 3, 2019 at 11:28
  • $\begingroup$ You need to base it on prior literature. I do not know what the Occurrences variable is, bit TimeStarted could be grouped by morning, afternoon, and night. You might try both dummy coding and making it a continuous variable and see what the outcome is. Just make sure to include this in your report. $\endgroup$
    – Jay Schyler Raadt
    Jun 3, 2019 at 12:17
  • $\begingroup$ Why not regress log(y) on the features, and, then take the exponent of the predicted value? From the sample you've shared, the turnover looks right-skewed so the log transform will have the additional benefit of making it "normal". $\endgroup$
    – StatsML
    Jun 19, 2019 at 5:30
  • $\begingroup$ Thanks @user2522806. Could you maybe provide an example in the form of an answer? $\endgroup$
    – jonboy
    Jun 19, 2019 at 5:32

1 Answer 1

Reset to default
2
+50
$\begingroup$

As suggested in the comment, you could try fit a model on the log of the labels and take the exponent of the predicted value

labels = np.log(labels)
X_train, X_test, y_train, y_test = train_test_split(features, labels, test_size = 0.3, random_state = 0)

# Create Linear Model
regressor = linear_model.LinearRegression()
regressor.fit(X_train, y_train)

# CoEfficient and Intercept
print(regressor.coef_)
print(regressor.intercept_)

# Predict the Output
y_Pred = regressor.predict(X_test)
print(np.exp(regressor.predict([[1, 0, 1, 0, 3]])))

The output is

[-4.77563348  0.52967564 -4.77563348  1.81217047 -1.02009026]
19.04716416404977
[623.67313587]
Improve this answer
$\endgroup$
1
  • $\begingroup$ Thanks @user2522806 $\endgroup$
    – jonboy
    Jun 19, 2019 at 6:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.