I am building a new metric for our organization. We ran a sample survey of 34 questions all on a five-point Likert scale, collected data, and used EFA to identify 10 survey questions that hang together well. Using Cronbach's Alpha for internal validity, we have a score of .89. I am hoping to gain some insight on results for our metric.
In the past, when we have built similar metrics, we have averaged each item's response by participant, and then averaged the averages across the organization to report our score for the survey. So if I answered 2, 3, 5, 2,and 3, I would have a personal score of 3.00. My personal score would be averaged with the scores of all other participants, to produce an organization-level score.
To my best understanding, this is the function of a validated Likert-scale (and not just Likert-type survey questions). We have some push-back this time around to only report out percent agree/percent disagree. In which case, I am not sure how to build this into a metric score. If I agree with 1/5 questions, am I a "20% agree", and all percent agrees are then averaged across the organization? If I agree with more statements than I disagree, do I get counted as an "In agreement" and we report how many participants in the organization fall in full agreement?
What say this community? There is lots of conversation on Likert-scales on the interwebs, but I have found less instruction on reporting out validated Likert-scale metrics.