I trained a linear regression model on some data. Now I have the intercept and the other coefficients. How to relate that with percent change in target given some percent change in a feature, keeping all others constant?
$\begingroup$
$\endgroup$
1
-
3$\begingroup$ Hi: unless the variables were both in logs when you did the regression, you can't do what you want to do. You can only get how many units the target will increase for every unit increase in the feature. $\endgroup$– mloftonJun 25, 2021 at 14:15
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
4
Let's say we have fitted a model such as:
$$ y = x_1 + x_2 + \epsilon$$
and we obtained esimates giving the following equation:
$$ \hat{y} = 2x_1 + 3x_2 $$
Thus for a 1 unit change in $x_1$ we expect a change of 2 units of $y$.
It is not possible to get the expected % change in $y$ from a % change in $x$ unless the variables are on a log scale prior to running the model.
answered Jun 25, 2021 at 14:42
-
$\begingroup$ An alternative interpretation is that the percent change conditional on the explanatory variables can be estimated. Regardless, what do the hats over the $x_i$ mean? $\endgroup$– whuber ♦Jun 25, 2021 at 14:56
-
$\begingroup$ @whuber the hats over the xs were a typo ! I think they are talking about estimating a % change in $y$ from a $p$% change in $x$ - I'm not sure how you would estimate that from the regression equation ? $\endgroup$ Jun 25, 2021 at 15:01
-
1$\begingroup$ An example of the alternative interpretation is a fitted model $\hat y=\hat\beta_0+\hat\beta_1x_1+\hat\beta_2x_2.$ Fixing $(x_1,x_2)$ and supposing $x_1\ne0,$ consider a $100p\%$ change in $x_1.$ The new estimate differs from the original by $\hat\beta_1(px_1),$ thereby inducing a $100\hat\beta_1(px_1)/\hat y$ percent change in $\hat y$ (assuming it, too, is nonzero). $\endgroup$– whuber ♦Jun 25, 2021 at 15:10
-
$\begingroup$ @RobertLong I understand this solution, and this will work to some extent. Two things I want to ask 1) Can I normalize the coefficients to 0-1 scale in such a way that their sum is 1. Thus each feature will contribute their normalised factor to the final value? 2)How does log help us here? $\endgroup$– kramerJun 26, 2021 at 9:48
$\begingroup$
$\endgroup$
4
We can back out the answer with a little auxiliary information about the covariates.
Your linear model is probably something like $$E[y \vert x, z] = \hat y= \hat \alpha + \hat \beta \cdot x +\hat \delta \cdot z.$$
What is the change in the expected value of $y$ associated with 1 unit change with $x$? We can easily get that from the derivative:$$\frac{\Delta \hat y}{\Delta x} =\hat \beta.$$
We can then turn that into an elasticity:
$$\epsilon =\frac{100\cdot\frac{\Delta \hat y}{\hat y}}{100 \cdot \frac{\Delta x}{x} }= \frac{\% \Delta \hat y}{\% \Delta x}= \hat \beta \cdot \frac{x}{\hat y}=\frac{\partial \hat y} {\partial x} \cdot \frac{x}{\hat y}.$$
In other words, to get an elasticity, we just need to multiply the regression coefficient on $x$ by the value of $x$ itself over the prediction of $y$. Note that this is a function of the covariates and can vary across observations. We need some way to summarize the individual elasticities.
There are several ways to put this into practice, but the most common is to take the average in the sample:
$$\bar \epsilon =\frac{1}{N} \sum_{i=1}^N \hat \beta \cdot \frac{x_i}{\hat y_i}.$$
Some software can do this for us, which makes the calculation of the standard errors much easier. Here's an example in Stata:
. sysuse auto, clear
(1978 automobile data)
. regress price mpg foreign
Source | SS df MS Number of obs = 74
-------------+---------------------------------- F(2, 71) = 14.07
Model | 180261702 2 90130850.8 Prob > F = 0.0000
Residual | 454803695 71 6405685.84 R-squared = 0.2838
-------------+---------------------------------- Adj R-squared = 0.2637
Total | 635065396 73 8699525.97 Root MSE = 2530.9
------------------------------------------------------------------------------
price | Coefficient Std. err. t P>|t| [95% conf. interval]
-------------+----------------------------------------------------------------
mpg | -294.1955 55.69172 -5.28 0.000 -405.2417 -183.1494
foreign | 1767.292 700.158 2.52 0.014 371.2169 3163.368
_cons | 11905.42 1158.634 10.28 0.000 9595.164 14215.67
------------------------------------------------------------------------------
. /* canned */
. margins, eyex(mpg)
Average marginal effects Number of obs = 74
Model VCE: OLS
Expression: Linear prediction, predict()
ey/ex wrt: mpg
------------------------------------------------------------------------------
| Delta-method
| ey/ex std. err. t P>|t| [95% conf. interval]
-------------+----------------------------------------------------------------
mpg | -1.238224 .3721885 -3.33 0.001 -1.980347 -.4961013
------------------------------------------------------------------------------
. /* by hand */
. predict yhat, xb
. generate elasticity = -294.1955 *(mpg/yhat)
. summarize elasticity
Variable | Obs Mean Std. dev. Min Max
-------------+---------------------------------------------------------
elasticity | 74 -1.238224 1.060581 -7.488722 -.4215304
This means that a 1% increase in mpg is associated with 1.2% decrease in price.
If we don't have the raw data, but have some summary statistics on the covariates and the coefficients, we can plug those into the first formula instead of averaging. The answers won't match exactly but are usually reasonably close.
-
2$\begingroup$ I just realized that @whuber suggested this very approach in the comment above. $\endgroup$– dimitriyJun 26, 2021 at 1:32
-
$\begingroup$ ϵ¯ is the overall elasticity. Just confirming that I won't need that for individual features, that's right? $\endgroup$– kramerJun 29, 2021 at 17:14
-
$\begingroup$ I am not sure I understand your question. Could you elaborate on what you mean by individual features? $\endgroup$– dimitriyJun 29, 2021 at 17:54
-
$\begingroup$ Had a little confusion. I got it now. Thanks. $\endgroup$– kramerJun 29, 2021 at 18:24