0
$\begingroup$

I am hoping to gain some insight on the most responsible way to fulfill a request at my place of work. I have been asked to show a quantifiable improvement year-over-year, by comparing apples and oranges. The data are unpaired - there is no way to link respondents from 2019 to 2020.

A survey was conducted in 2019 to measure internal customer satisfaction metrics, and asked a series of 8 questions with a 0 - 10 satisfaction scale, such as:

  1. Department A are responsive to requests for information.
  2. Department A staff are open, honest, and communicative.
  3. Department A consistently does good work. etc...

A survey was conducted in 2020 and sent to roughly the same cohort (all-staff, +/- staff who have left or joined the company). Instead of asking 8 questions, only two questions were asked with the same 0 - 10 satisfaction scale:

  1. My satisfaction level for the services provided by Department A is:
  2. My satisfaction level with the staff from Department A is:

More specifically, after indicating that the respondent has worked with Department A over the past year, the question is worded as: "How satisfied are you with the services offered by Department A?". 0 is labeled as "Not at all satisfied" and 10 is labeled "Extremely satisfied". All other scale points are not labeled.

What is the most responsible way to show a quantitative comparison or difference between years?

Besides cautioning that this was not the intention of the survey design, are there any statistical tests that could be run with caution? Any insight is greatly appreciated.

$\endgroup$
2
  • $\begingroup$ Forget statistical testing--by posting this question, you show that (at least intuitively) you understand that a valid comparison of these two surveys is impossible. Even without the issue of comparing two surveys, the value of the second survey in itself is questionable. How is a "satisfaction level" defined and understood by the respondents? $\endgroup$ – whuber Jan 20 at 23:32
  • 1
    $\begingroup$ The "satisfaction level" from the second survey was designed to be a proxy net promoter score question... The question is a bit more descriptive than what I have above. After a respondent selects that they have worked with the specific department, they are then asked "How satisfied are you with the services offered by Department A?". 0 is labeled as "Not at all satisfied" and 10 is labeled "Extremely satisfied". All other scale points are not labeled. $\endgroup$ – Matt Jan 20 at 23:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.