When running a regression, statistical significance of an effect is important but its magnitude is even more important.
How should I evaluate the size of an effect? I've read that usually in Econometrics, the standard error is used as a benchmark but how?
This is an example of a regression table I've to analyze (Miguel, Kremer Econometrica (2004)). How should I see if the effect of "Indicator from Group 1 School" is high or not?
Maybe an example will be helpful. This very simple example is from Gelman and Hill (2006, p.31-34). We want to predict cognitive test scores of children (kid.score) given their mothers' education (mom.hs) and IQ (mom.iq): mom.hs is a binary predictor indicating whether mother graduated from high school (1) or not (0), and mom.iq is a continuous predictor. The fitted linear regression model is
$$\text{kid.score} = 26+6\cdot \text{mom.hs}+0.6 \cdot \text{mom.iq} + \text{error} $$
Now, the interpretation is rather straightforward. For example, for 1 unit increase in mom.iq, I expect 0.6 points increase in kid.score (keeping the value of mom.hs constant). This relationship (between kid.score and mom.iq) is statistically significant, but is it practically significant? Is the effect size, here unstandardized regression coefficient (0.6), large? Actually, I don't know. I need to have an idea about the distribution of the values for dependent and independent variables. But more importantly, I need to know about the theories explaining the relationship between cognitive test score of children and the maternal IQ.
Looking at the table and assuming this is a linear model (and unstandardized regression coefficients are reported), for 1 unit increase in "Indicator for Group 1 (1998 Treatment) School", I expect to see 0.25 points decrease in dependent variable (for model 1 and holding other variables constant). Again, I need to know theories explaining the relationship between variables, which will guide me in my interpretation of the magnitude of the effects.
I can't think of a benchmark in this context. But, if one function of having a benchmark is to make comparisons possible, standardized coefficients might be worth considering. Standardized (beta, $\hat\beta*$) coefficients are more easily comparable, well, because the variables are standardized to have a mean of 0 and standard deviation 1. You can compare beta coefficients (in standard deviation units) to assess the relative strength of the predictors, for example "One standard deviation increase/decrease in X would yield a $\hat\beta*$ standard deviation increase/decrease in Y". Acock (2014) also argues that they can be interpreted similar to correlations: $\hat\beta* < 0.2$ is considered a weak, $0.2 < \hat\beta* < 0.5$ moderate, and $\hat\beta* > 0.5$ strong effect (p.272), but I can't verify this information from another source. Moreover, I come up with some (strong) warnings about the use of standardized coefficients (Fox, 2016, p.102; Harrell, 2015, p.103-104), this and this are examples.
So, repeating once more, to evaluate the size of an effect (based on this output, unstandardized regression coefficients), you need to have information about the variables (e.g., how they are measured, their distributions, range of values, etc.), and the theories explaining the relationship between them.
Acock, A. C. (2014). A Gentle Introduction to Stata (4th ed.). Texas: Stata Press.
Fox, J. (2016). Applied Regression Analysis and Generalized Linear Models (3rd ed.). Los Angeles: Sage Publications.
Gelman, A., & Hill, J. (2006). Data Analysis Using Regression and Multilevel Models. Cambridge: Cambridge University Press.
Harrell, F. E. (2015). Regression Modeling Strategies (2nd ed.). Cham: Springer.
