My stats knowledge is average but not great, so I apologise if this is something I just haven't come across before. Basically, I've just read a paper which seems to state that it has conducted correlational analyses on a set of variables, where the IV scores are not matched with the DV scores, and I simply don't see how this is possible? To give more detail:
The study is looking at the possible association between:
a) anti-social personality traits, and social climate on a therapeutic prison unit
b) suicidal ideation, and social climate on a therapeutic prison unit
The measures were operationalised and measured in the following way:
The independent variables (anti-social personality and suicidal ideation) were measured via a personality inventory given to everybody entering the therapeutic unit.
The dependent variable (social climate) was measured using an established questionnaire of social climate, comprising three sub scales. The questionnaire was given out to everyone who had completed the IV measures, BUT responses were returned anonymously. The final sample size was 125.
Therefore, there was no way of matching the social climate scores to the scores for the IV measures.
My issue is that I don't see how any tests of correlation can possibly work, if you don't have an IV score AND a matched DV score for each participant?
The description of the analytical procedures from the article might help. It says (paraphrased for ease of reading):
"As the [social climate measure] was completed anonymously, the data from the two questionnaires were unmatched. Therefore, data was grouped by [the five different groups within the therapeutic unit], producing five sets of data."
"Kendall’s tau-b rank correlation coefficient analysis was administered to explore correlations between variables. This was appropriate as the data was not normally distributed, non-linear, and the sample size was relatively small [125 residents]. Kendall’s correlation is a robust and efficient non-parametric test (Croux & Dehon, 2010). Mean scores for each measure were used in the analysis because the raw data was unmatched."
A table of correlations between the IVs and all the DV sub scales is then presented, which looks just like a regular correlation table.
The first comment (about splitting the data into five groups) I just don't understand why that makes a difference? The second comment about Kendall's tau I can understand because of other characteristics of the data. But the last part about using mean scores, I don't understand either. What does that look like and how does it help?
The only way I can make any sense of it (and this isn't detailed in the article, this is purely a guess), is that they took the mean IV and DV score for each of the five therapy groups within the bigger therapy unit, which would produce five pairs of matched scores.
But even with those "matched" scores, that would be effectively a sample size of 5, right? Which is way too small for any correlational analyses to be reliable. Plus by using the means, wouldn't that obscure information on variance, which is needed for correlational analyses? I'm not even sure if this is what they did anyway.
Basically my main question is: is this bad statistics, or is this me just not understanding what on earth is going on??
Would really appreciate some answers in not too complicated language :)