I assessed the internal reliability of a self-created scale with eight items (N total = 150) by calculating Cronbach’s α. It appears that one item correlates low with the overall score of the scale (item 4 in the example below). The corrected item-total correlation, i.e. the correlation of this item with the scale total excluding that item, is only r= .046.
library(psych)
scale<-mydata[,c(24,25,26,27,28,29,30,31)]
alpha(scale)
Reliability analysis
Call: alpha(x = scale)
0.62 0.64 0.66 0.18 4.3 0.79
Reliability if an item is dropped:
raw_alpha std.alpha G6(smc) average_r
item1 0.56 0.59 0.62 0.17
item2 0.53 0.57 0.58 0.16
item3 0.54 0.56 0.58 0.16
item4 0.66 0.67 0.67 0.23
item5 0.60 0.62 0.63 0.19
item6 0.55 0.59 0.62 0.17
item7 0.58 0.61 0.63 0.18
item8 0.63 0.65 0.67 0.21
Item statistics
n r r.cor r.drop mean sd
item1 144 0.60 0.51 0.395 4.5 0.71
item2 145 0.65 0.62 0.499 4.6 0.71
item3 142 0.67 0.64 0.484 4.5 0.72
item4 146 0.33 0.15 0.046 4.6 0.81
item5 147 0.51 0.41 0.298 4.9 0.41
item6 139 0.59 0.50 0.404 4.4 0.82
item7 136 0.53 0.43 0.339 4.2 1.03
item8 135 0.39 0.21 0.190 4.3 0.94
Non missing response frequency for each item
1 2 3 4 5 miss
item1 0.01 0.01 0.04 0.34 0.60 0.04
item2 0.01 0.01 0.03 0.24 0.71 0.03
item3 0.00 0.01 0.11 0.28 0.60 0.05
item4 0.01 0.03 0.05 0.14 0.77 0.03
item5 0.00 0.00 0.02 0.11 0.87 0.02
item6 0.01 0.02 0.10 0.29 0.58 0.07
item7 0.04 0.02 0.15 0.25 0.54 0.09
item8 0.02 0.03 0.11 0.32 0.52 0.10
PROBLEM: I would like to report this low correlation with the degrees of freedom in parentheses and the significance level in the main text. Yet, I am not sure whether I calculated the correct p-value. What I did is a simple regression with item 4 as the dependent variable:
scale <- as.data.frame(scale)
summary(lm(item4 ~ item1+item2+item3+item5+item6+item7+item8, data=scale))
Call:
lm(formula = item4 ~ item1 + item2 + item3 + item5 + item6 +
item7 + item8, data = scale)
Residuals:
Min 1Q Median 3Q Max
-3.4256 -0.0465 0.2869 0.3500 1.3405
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 1.65098 1.00288 1.646 0.102492
item1 0.12916 0.11560 1.117 0.266262
item2 0.02387 0.12921 0.185 0.853760
item3 -0.07323 0.12718 -0.576 0.565921
item5 0.64204 0.18636 3.445 0.000802 ***
item6 -0.04596 0.10230 -0.449 0.654120
item7 -0.13217 0.08030 -1.646 0.102545
item8 0.05609 0.08758 0.641 0.523136
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Residual standard error: 0.8235 on 113 degrees of freedom
(29 observations deleted due to missingness)
Multiple R-squared: 0.1385, Adjusted R-squared: 0.08518
F-statistic: 2.596 on 7 and 113 DF, p-value: 0.01604
QUESTION: Is it correct if I report something like "Item 4 correlates only weakly with the overall score of the scale (r(113)= .046, p= .02)" -- or did I make a rather large error in reasoning here?
Many thanks!
