5
$\begingroup$

I am working with spoken language data and use linear models do determine the relationship between different phonological processes in my data.

Background

Measures of the regularity of syllable durations are influenced by how syllable is defined. I used two methods to determine syllable boundaries. Method 1 and 2 give different results for regularity of syllable durations, but the difference between the two methods is not the same for every speaker.

Linear model

I think the frequency of certain phonological processes (vowel elision, insertion of glottal stops and others) might be responisble for these differences. I want to find out if that's true and if yes how much each of these phonological processes contributes to explaining the differences.

Measuring effect size with partial $R^2$/sum of squares

To measure effect size, I could use partial $R^2$. The data below is the ANOVA of my model, and from the variance explained by each factor (Sum Sq) I can derive how much of the total variance each factor explains.

Analysis of Variance Table

Response: varcos
                          Df Sum Sq Mean Sq F value    Pr(>F)    
rel_gs                     1 395.48  395.48 46.9135 7.022e-07 ***
rel_all2_gs                1   9.47    9.47  1.1236    0.3006    
rel_gs:rel_all_gs          1   1.37    1.37  0.1622    0.6910    
rel_gs:rel_all2_gs         1  13.63   13.63  1.6168    0.2168    
rel_all_gs:relDur.mean.y   1  15.94   15.94  1.8910    0.1829    
rel_all2_gs:relDur.mean.y  1  69.19   69.19  8.2079    0.0090 ** 
Residuals                 22 185.46    8.43                      

Coeffcients

Beta coefficients can also be interpreted as a measure of effect size, as an answer to this question points out. I was expecting that factors that explain a large part of the variance would also have large coeffcients, and the other way around. But that's not the case. rel_gs:rel_all2_gs has a huge negative coefficient, but explains only a small part of total variance.

Call:
lm(formula = varcos ~ rel_gs + rel_all2_gs + rel_all_gs:rel_gs + 
    rel_all2_gs:rel_gs + relDur.mean.y:rel_all_gs + relDur.mean.y:rel_all2_gs, 
    data = read.wwb3.diff_syll.perc)

Residuals:
    Min      1Q  Median      3Q     Max 
-5.7929 -1.5933 -0.5208  1.5569  5.4834 

Coefficients:
                           Estimate Std. Error t value Pr(>|t|)    
(Intercept)                  15.539      2.725   5.702  9.8e-06 ***
rel_gs                     -336.020    118.306  -2.840  0.00952 ** 
rel_all2_gs                 492.997    183.440   2.688  0.01345 *  
rel_gs:rel_all_gs          2533.220    810.012   3.127  0.00490 ** 
rel_gs:rel_all2_gs        -2292.357    716.625  -3.199  0.00414 ** 
rel_all_gs:relDur.mean.y   -955.239    315.827  -3.025  0.00623 ** 
rel_all2_gs:relDur.mean.y   691.578    241.394   2.865  0.00900 ** 

Question

Should I only trust partial $R^2$ for effect size and ignore differences between the coefficients?

P.S.: I asked a related question some time ago, and it hasn't received much attention and left some issues open. I suppose that might be because the original question wasn't clear enough or was based on toy data. In this question I'm using actual data. If my question could be imporoved in some way I'd be happy to consider any suggestions.

$\endgroup$
0
4
$\begingroup$

Partial R² is what you should be looking for. The partial R² is the proportion of variance explained in the dependent variable by a given predictor, controlling for the other predictors in the model. The partial R²s can be compared to establish the relative strength of the predictors in your model. Alternatively, you can be looking at standardized regression coefficients, which represent the expected change in the dependent variable for each increase of 1 standard deviation on the predictor. The standardized coefficients are often used to compare the relative strength of predictors in a single model.

The unstandardized regression coefficients, which are what you are looking at in your question, represent the expected change in the dependent variable for each 1-unit increase in the predictor, holding all other predictors constant. Therefore, the scaling of each regression coefficient depends in part on the strength of the relationship between the predictor and the dependent variable, but also on the scaling of the predictor (a 1-unit increase on income in dollars does not represent the same change as a 1-unit increase on income in thousands of dollars, for example). Therefore, unless the predictors are all on the same scale (and have the same variability), you should not be comparing the unstandardized regression coefficients with each other as a way to establish the relative predictive power of the predictors in your model.

$\endgroup$
2
  • 1
    $\begingroup$ Thanks for your answer! I applied scaling in this question and there were still substantial differences between the size of the coefficients and partial $R^2$. Are there cases when you think scaling is not appropriate? $\endgroup$ – robert Jul 29 '13 at 16:55
  • $\begingroup$ I don't see the usefulness of changing the scale of your predictors; you're losing interpretability without gaining much in return. If there are substantial differences in partial R²s, then it means that some of your predictors are substantially better predictors of your DV than others, controlling (holding constant) all the other predictors in your model. Remember that when you have interaction terms, it changes the interpretation of the coefficient of any predictor involved in an interaction (see my answer here: tinyurl.com/md7n8fg). This problem is sidestepped when using partial R². $\endgroup$ – Patrick Coulombe Jul 29 '13 at 17:03

Not the answer you're looking for? Browse other questions tagged or ask your own question.