Significance Test for Comparison of Variables

Question

I am not sure how to ask this question without giving an example.

I am trying to measure the "cleanliness" of office buildings. I have two variables that try to measure this.

Variable one is a person who inspects a random sample of the offices in a building and rates the cleanliness of each office. These scores are averaged together to get an overall score for the building.

Variable two is the number of janitorial requests to clean up a messy area in a building. Someone actually calls the maintenance office and says "there is a mess in this office can someone with a broom/mop come clean it up." This is the exact count at the building level, not a sample.

The idea is that both variables are trying to measure the same concept, "cleanliness." As such, I expect some "congruity" between the two variables. The way I am trying to measure the congruity is by using Spearman's Correlation (ranking the buildings by cleanliness for each variable and applying the correlation formula to the ranks).

Here are my questions: 1) Can Spearman's correlation coefficient be used as an indicator of whether or not the inspectors are accurately scoring the offices (assuming janitorial requests are an appropriate measure of cleanliness).

2) Does the fact that the building's score from the inspector is a function of a sample of offices from the building while the number of janitorial requests is a population measure affect the calculation of correlation coefficient?

3) If Spearman's correlation is an OK measure to use, what range of values would be considered "Strong", "Medium", "Weak"? Is this up to the end user of the data?

4) If I am only concerned with 10 buildings and I have data for all 10 buildings, do I need to use significance tests? For example, is the coefficient different from zero? My understanding of significance tests is that they are needed when data is being sampled from a population but not when you have the population level of data.

Answer 1

I guess your two variables does not really measure the same thing. The subjectiv valuation of "cleanliness" probably will score something for "not clean" way before the level is so bad that somebody calls the janitor. You should tell us something more about for what you will use your results, and you should post here (edit your question text) some plots of the two variables against each other. When you have posted such clarification, I or somebody else will be able to give more specific advice! One plot for each of the ten buildings, for instance (you do not need to post all of them, but you should comment if they look alike). Ooops! you do not seem to have paired data (do you?) you could also post some part of your data to make the structure clearer! (This is really more of a comment, but too long to post as a comment)

Answer 2

The general significance test for correlations is to see if the correlation is different from 0, which is probably not very interesting in your case. Meaningful correlations can be non-significant and correlations so small as to be meaningless can be significant.

What is probably more interesting than seeing if they are correlated, is investigating when they are not as correlated as expected. One approach is to use Bland-Altman plots (https://en.wikipedia.org/wiki/Bland%E2%80%93Altman_plot) or other diagnostic tools to fully explore the relationship and deviations from the expected relationship.