Non-Significant difference between individual items, but significant difference between overall scores Say that we have a scale measuring Healthiness of Diet, and want to see if two populations have different levels of Healthiness of Diet. 
If we find that the difference between the two samples at the individual item levels are almost all non-significant (e.g., most p values are around >4, one or two out of 20 items have a 0.001 correlation), but the overall score is significant (around p=0.03), how might we interpret this result?
Here's a specific example, looking at healthiness of diets (Zhang et al., 2015):

Sure it's significant overall, but something doesn't feel quite right; feels like a fairly weak finding. How would you approach this finding? 
 A: When you combine questions you increase the reliability of the measure, and with a more reliable measure, you have more power. 
The increase in reliability that you get from more questions is estimated using the Spearman Brown prophecy formula, here's  the Wikipedia page: https://en.wikipedia.org/wiki/Spearman%E2%80%93Brown_prediction_formula (Oh, it calls it the prediction formula; same thing).  
Here's how to think about this. The variance of the sum of two uncorrelated variables is equal to the sum of their variances. The error of each question is random (by definition), so the error standard deviation increases with the square root of the number of items. But the effect size is the same - it's correlated 1, so the difference between the groups increases linearly with the number of items. The difference between the means is getting larger faster than the standard deviation is getting larger, so the effect size is increasing.
You can also think of this in terms of information. I have a group of people, and I ask them some questions. I can increase my power (and my probability of a significant result) by increasing the number of questions that I ask.
Here's a practical example. I have two groups, and an outcome which has (population) sd=1. The difference between the two groups is 0.1. In a sample of 100, I have very little power to detect that effect. 
Check R, and it appears my power is about 0.10. 
> power.t.test(d=0.1, n=100)

     Two-sample t test power calculation 

              n = 100
          delta = 0.1
             sd = 1
      sig.level = 0.05
          power = 0.104507
    alternative = two.sided

NOTE: n is number in *each* group

But what if I had 100 questions, all with the same effect size? And I will sum these items. The variance will be equal to the variance (which is 1) times 100 = 100, so the standard deviation will be 10. 
But what of the difference? We just add that up - so the expected difference between the groups is 100 - 0.1, which is equal to 10. Now our expected difference is equal to the standard deviation.  Plug that into a power calculation and our power is 0.99999.
> power.t.test(d=1, n=100)

     Two-sample t test power calculation 

              n = 100
          delta = 1
             sd = 1
      sig.level = 0.05
          power = 0.9999998
    alternative = two.sided

NOTE: n is number in *each* group

That's the theory, let's try it in practice:
set.seed <- 1234
#create empty data frame
df <- as.data.frame(matrix(rnorm(10000), nrow=100))  
#add names
names(df) <-paste0("y", 1:100)
#create group variable
df$group <- c(rep(0, 50), rep(1, 50))

#Create difference score
es <- 0.1
df[,paste0("y", 1:100)] <- apply(df[,paste0("y", 1:100)], 2, function(x) x + es * df$group)

#do some t-tests
t.test(df$y1 ~ df$group)
t.test(df$y2 ~ df$group)

#We're going to get bored before we do them all. Let's just get the p-values.
pValues <- apply(df[,paste0("y", 1:100)], 2, function(x) t.test(x ~ df$group)$p.value)

#How many are sig?
sum(pValues < 0.05)

What do we get?
> sum(pValues < 0.05)
[1] 11

11 - very close to the 10 that we expected.
But now let's get the sum of the items:
#Calculate the total
df$total <- apply(df[,paste0("y", 1:100)], 1, sum)

And do a t-test:
#Now t-test on total
t.test(df$total ~ df$group)

    Welch Two Sample t-test

data:  df$total by df$group
t = -7.1753, df = 95.758, p-value = 1.527e-10
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 -16.020834  -9.077376
sample estimates:
mean in group 0 mean in group 1 
      -1.418933       11.130173 

Monstrously significant, as we expected,  the difference is close to 10 (as we expected) and the confidence intervals include 10. 
It's also possible to test whether the size of the effect is different for different items - I'm assuming it's always the same. But that's another question. :)
