Skip to main content
Edited title and text for clarity & removed tags
Source Link
mkt
  • 20.4k
  • 11
  • 81
  • 187

OLS R-Squared Drops by 15% When Constant / Intercept Why does my model $R^2$ drop when the intercept term is Removedremoved?

I ran an Ordinary Least Squares model and found the constant  / interceptintercept is the more significant than all the other features. When constantthe intercept is included, the R-squared$R^2$ is 45%, when. When I remove the constantintercept, the R-squared$R^2$ drops to 29%.

The constantintercept also has the lowest p-value compared to all the other features.

Moreover, I used StandardScaler to scale the features used.

Why would the intercept be so significant?

Example code:

model_2 = sm.OLS(df_reg_y.astype(float), sm.add_constant(X_scaled.astype(float))).fit()

This area circled in purple is the scatter plot of the target variable vs the most significant feature. The histogram below it is the distribution of the target variable.

enter image description here

Thanks in advance!

Edit: I realized that I forgot to scale the target variable. The issue was fixed after I scaled the features and target variable together.

OLS R-Squared Drops by 15% When Constant / Intercept is Removed?

I ran an Ordinary Least Squares model and found the constant  / intercept is the more significant than all the other features. When constant is included, the R-squared is 45%, when I remove the constant, the R-squared drops to 29%.

The constant also has the lowest p-value compared to all the other features.

Moreover, I used StandardScaler to scale the features used.

Why would the intercept be so significant?

Example code:

model_2 = sm.OLS(df_reg_y.astype(float), sm.add_constant(X_scaled.astype(float))).fit()

This area circled in purple is the scatter plot of the target variable vs the most significant feature. The histogram below it is the distribution of the target variable.

enter image description here

Thanks in advance!

Edit: I realized that I forgot to scale the target variable. The issue was fixed after I scaled the features and target variable together.

Why does my model $R^2$ drop when the intercept term is removed?

I ran an Ordinary Least Squares model and found the constant/intercept is more significant than all the other features. When the intercept is included, the $R^2$ is 45%. When I remove the intercept, the $R^2$ drops to 29%.

The intercept also has the lowest p-value compared to all the other features.

Moreover, I used StandardScaler to scale the features used.

Why would the intercept be so significant?

Example code:

model_2 = sm.OLS(df_reg_y.astype(float), sm.add_constant(X_scaled.astype(float))).fit()

This area circled in purple is the scatter plot of the target variable vs the most significant feature. The histogram below it is the distribution of the target variable.

enter image description here

Edit: I realized that I forgot to scale the target variable. The issue was fixed after I scaled the features and target variable together.

added 409 characters in body; edited tags
Source Link
SAKURA
  • 205
  • 2
  • 8

I ran an Ordinary Least Squares model and found the constant / intercept is the more significant than all the other features. When constant is included, the R-squared is 45%, when I remove the constant, the R-squared drops to 29%.

The constant also has the lowest p-value compared to all the other features.

Moreover, I used StandardScaler to scale the features used.

Why would the intercept be so significant?

Example code:

model_2 = sm.OLS(df_reg_y.astype(float), sm.add_constant(X_scaled.astype(float))).fit()

This area circled in purple is the scatter plot of the target variable vs the most significant feature. The histogram below it is the distribution of the target variable.

enter image description here

Thanks in advance!

Edit: I realized that I forgot to scale the target variable. The issue was fixed after I scaled the features and target variable together.

I ran an Ordinary Least Squares model and found the constant / intercept is the more significant than all the other features. When constant is included, the R-squared is 45%, when I remove the constant, the R-squared drops to 29%.

The constant also has the lowest p-value compared to all the other features.

Moreover, I used StandardScaler to scale the features used.

Why would the intercept be so significant?

Example code:

model_2 = sm.OLS(df_reg_y.astype(float), sm.add_constant(X_scaled.astype(float))).fit()

I ran an Ordinary Least Squares model and found the constant / intercept is the more significant than all the other features. When constant is included, the R-squared is 45%, when I remove the constant, the R-squared drops to 29%.

The constant also has the lowest p-value compared to all the other features.

Moreover, I used StandardScaler to scale the features used.

Why would the intercept be so significant?

Example code:

model_2 = sm.OLS(df_reg_y.astype(float), sm.add_constant(X_scaled.astype(float))).fit()

This area circled in purple is the scatter plot of the target variable vs the most significant feature. The histogram below it is the distribution of the target variable.

enter image description here

Thanks in advance!

Edit: I realized that I forgot to scale the target variable. The issue was fixed after I scaled the features and target variable together.

deleted 18 characters in body; edited tags
Source Link
Ferdi
  • 5.3k
  • 10
  • 47
  • 64

I ran an Ordinary Least Squares model and found the constant / intercept is the more significant than all the other features. When constant is included, the R-squared is 45%, when I remove the constant, the R-squared drops to 29%.

The constant also has the lowest p-value compared to all the other features.

Moreover, I used StandardScaler to scale the features used.

Why would the intercept be so significant?

Example code:

model_2 = sm.OLS(df_reg_y.astype(float), sm.add_constant(X_scaled.astype(float))).fit()

Thanks in advance!

I ran an Ordinary Least Squares model and found the constant / intercept is the more significant than all the other features. When constant is included, the R-squared is 45%, when I remove the constant, the R-squared drops to 29%.

The constant also has the lowest p-value compared to all the other features.

Moreover, I used StandardScaler to scale the features used.

Why would the intercept be so significant?

Example code:

model_2 = sm.OLS(df_reg_y.astype(float), sm.add_constant(X_scaled.astype(float))).fit()

Thanks in advance!

I ran an Ordinary Least Squares model and found the constant / intercept is the more significant than all the other features. When constant is included, the R-squared is 45%, when I remove the constant, the R-squared drops to 29%.

The constant also has the lowest p-value compared to all the other features.

Moreover, I used StandardScaler to scale the features used.

Why would the intercept be so significant?

Example code:

model_2 = sm.OLS(df_reg_y.astype(float), sm.add_constant(X_scaled.astype(float))).fit()

Source Link
SAKURA
  • 205
  • 2
  • 8
Loading