71
$\begingroup$

I have some ordinal data gained from survey questions. In my case they are Likert style responses (Strongly Disagree-Disagree-Neutral-Agree-Strongly Agree). In my data they are coded as 1-5.

I don't think means would mean much here, so what basic summary statistics are considered usefull?

Improve this question
$\endgroup$
1
  • 5
    $\begingroup$ Common choices include - medians, modes, proportions or cumulative proportions in each group $\endgroup$
    – Glen_b
    Commented Jan 22, 2014 at 5:59

8 Answers 8

Reset to default
30
$\begingroup$

A frequency table is a good place to start. You can do the count, and relative frequency for each level. Also, the total count, and number of missing values may be of use.

You can also use a contingency table to compare two variables at once. Can display using a mosaic plot too.

Improve this answer
$\endgroup$
36
$\begingroup$

I'm going to argue from an applied perspective that the mean is often the best choice for summarising the central tendency of a Likert item. Specifically, I'm thinking of contexts such as student satisfaction surveys, market research scales, employee opinion surveys, personality test items, and many social science survey items.

In such contexts, consumers of research often want answers to questions like:

  • Which statements have more or less agreement relative to others?
  • Which groups agreed more or less with a given statement?
  • Over time, has agreement gone up or down?

For these purposes, the mean has several benefits:

1. Mean is easy to calculate:

  • It is easy to see the relationship between the raw data and the mean.
  • It is pragmatically easy to calculate. Thus, the mean can be easily embedded into reporting systems.
  • It also facilitates comparability across contexts, and settings.

2. Mean is relatively well understood and intuitive:

  • The mean is often used to report central tendency of Likert items. Thus, consumers of research are more likely to understand the mean (and thus trust it, and act on it).
  • Some researchers prefer the, arguably, even more intuitive option of reporting the percentage of the sample answering 4 or 5. I.e., it has the relatively intuitive interpretation of "percentage agreement". In essence, this is just an alternative form of the mean, with 0, 0, 0, 1, 1 coding.
  • Also, over time, consumers of research build up frames of reference. For example, when you're comparing your teaching performance from year to year, or across subjects, you build up a nuanced sense of what a mean of 3.7, 3.9, or 4.1 indicates.

3. The mean is a single number:

  • A single number is particularly valuable, when you want to make claims like "students were more satisfied with Subject X than Subject Y."
  • I also find, empirically, that a single number is actually the main information of interest in a Likert item. The standard deviation tends to be related to the extent to which the mean is close to the central score (e.g., 3.0). Of course, empirically, this may not apply in your context. For example, I read somewhere that when You Tube ratings had the star system, there were a large number of either the lowest or the highest rating. For this reason, it is important to inspect category frequencies.

4. It doesn't make much difference

  • Although I have not formally tested it, I would hypothesise that for the purpose of comparing central tendency ratings across items, or groups of participants, or over time, any reasonable choice of scaling for generating the mean would yield similar conclusions.
Improve this answer
$\endgroup$
4
  • 4
    $\begingroup$ Nice post! Do you have any thoughts on how different cultures / countries may use likert scales which would drastically impact these sorts of results? $\endgroup$
    – Chase
    Commented Aug 13, 2011 at 15:03
  • $\begingroup$ @chase There is research on this, but it's been a while since I've looked at it. Here's an example search on Google scholar scholar.google.com.au/… $\endgroup$
    – Jeromy Anglim
    Commented Aug 14, 2011 at 3:58
  • $\begingroup$ I agree with Mr. Jeromy Anglim's justification on the use of mean (i.e., weighted mean to be exact) as the most reliable unbiased more descriptive interpretation of ordered categorical variables such as the use of Likert scale where each of the data points contribute to the final average. $\endgroup$
    – user9951
    Commented Mar 19, 2012 at 6:16
  • 3
    $\begingroup$ I asked the question about cultural differences in use of response scales on cogsci.stackexchange.com $\endgroup$
    – Jeromy Anglim
    Commented Jun 8, 2012 at 2:05
29
$\begingroup$

For basic summaries, I agree that reporting frequency tables and some indication about central tendency is fine. For inference, a recent article published in PARE discussed t- vs. MWW-test, Five-Point Likert Items: t test versus Mann-Whitney-Wilcoxon.

For more elaborated treatment, I would recommend reading Agresti's review on ordered categorical variables:

Liu, Y and Agresti, A (2005). The analysis of ordered categorical data: An overview and a survey of recent developments. Sociedad de Estadística e Investigación Operativa Test, 14(1), 1-73.

It largely extends beyond usual statistics, like threshold-based model (e.g. proportional odds-ratio), and is worth reading in place of Agresti's CDA book.

Below I show a picture of three different ways of treating a Likert item; from top to bottom, the "frequency" (nominal) view, the "numerical" view, and the "probabilistic" view (a Partial Credit Model):

alt text

The data comes from the Science data in the ltm package, where the item concerned technology ("New technology does not depend on basic scientific research", with response "strongly disagree" to "strongly agree", on a four-point scale)

Improve this answer
$\endgroup$
14
$\begingroup$

Conventional practice is to use the non-parametric statistics rank sum and mean rank to describe ordinal data.

Here's how they work:

Rank Sum

  • assign a rank to each member in each group;

  • e.g., suppose you are looking at goals for each player on two opposing football teams then rank each member on both teams from first to last;

  • calculate rank sum by adding the ranks per group;

  • the magnitude of the rank sum tells you how close together the ranks are for each group

Mean Rank

M/R is a more sophisticated statistic than R/S because it compensates for unequal sizes in the groups you are comparing. Hence, in addition to the steps above, you divide each sum by the number of members in the group.

Once you have these two statistics, you can, for instance, z-test the rank sum to see if the difference between the two groups is statistically significant (I believe that's known as the Wilcoxon rank sum test, which is interchangeable, i.e., functionally equivalent to the Mann-Whitney U test).

R Functions for these statistics (the ones I know about, anyway):

wilcox.test in the standard R installation

meanranks in the cranks Package

Improve this answer
$\endgroup$
3
$\begingroup$

I usually like to use Mosaic plot. You can create them by incoorporating other covariates of interest (such as: sex, stratified factors etc.)

Improve this answer
$\endgroup$
3
$\begingroup$

Based on the abstract This article may be helpful for comparing several variables that are Likert scale. It compares two types of non-parametric multiple comparison tests: One based on ranks and one based on a test by Chacko. It includes simulations.

Improve this answer
$\endgroup$
2
  • $\begingroup$ At present, this almost seems like a comment, @PeterFlom. Although the ACM digital library is probably less susceptible to link rot, would you mind saying something about the article, perhaps a precis of the helpful info it provides? $\endgroup$
    – gung - Reinstate Monica
    Commented Sep 2, 2012 at 17:42
  • 2
    $\begingroup$ Hi @gung I wasn't sure where to put the comment in this long thread. I added the cite here since the question posted today was closed, and this article seems useful (and covers issues I haven't seen elsewhere) $\endgroup$
    – Peter Flom
    Commented Sep 2, 2012 at 18:25
2
$\begingroup$

I agree with Jeromy Anglim's evaluation. Remember that Likert responses are estimates — you are not using a perfectly reliable ruler to measure a physical object with stable dimensions. The mean is a powerful measure when using reasonable sample sizes.

In business and product R&D, the mean is by far the most common statistic used with Likert scales. When using Likert scales I have usually chosen a measure that ideally fits the research question. For instance, if you are talking about "preference" or "attitudes" you can use multiple Likert-based indicators, with each indicator providing slightly different insight.

To evaluate the question "how will people in segment $i$ react to service offering $X$," I may look at (1) arithmetic mean, (2) exact median, (3) percentage most favorable response (top box), (4) % top two boxes, (5) ratio of top two boxes to bottom two boxes, (6) percentage within mid-range boxes... etc. Each measure tells a different piece of the story. In a very critical project, I use multiple Likert-based indicators. I will also use multiple indicators with small samples and when a specific cross tab has an "interesting" structure or looks information-rich. Ahhh... the art of statistics.

Improve this answer
$\endgroup$
1
$\begingroup$

"Box scores" are often used to summarize ordinal data, particularly when it comes with meaningful verbal anchors. In other words, you might report "top 2 box", the percentage that chose either "agree" or "strongly agree".

Improve this answer
$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.