Recommendations - or best practices - for analyzing non-independent data. Specific example relating to pain perception data provided

Question

I am seeking recommendations and/or best practices for analyzing non-independent data. In particular, I am curious about non-independent data that does not reflect typical repeated-measures time-based data in which data for the same question(s) or stimuli are collected at different time points. Rather the data collected is elicited from similar (but not identical) questions or stimuli that are known to be related. A specific example follows.

I have pain perception data for two groups (group A, group B) that can be further divided by gender (i.e., A-women; A-men; B-women; B-men). The pain perception dependent variables include 3 thermal threshold tests (i.e., detection, pain, tolerance) and pain magnitude estimates for a range of specific temperatures (temp X1, X2, X3, X4, X5). These pain perception variables are collected for both heat and cold stimuli (e.g., heat pain tolerance; cold pain tolerance; heat temp X1; cold temp X1). It should be expected (and indeed it is observed) that the various pain perception variables for a given individual are not independent.

The purpose of the analysis is to look for between group differences based on group membership (group A vs. group B) and sex (women vs men). It is desirable to find interactions between group membership and sex. It is also desirable to find interactions between specific pain perception variables (or types of variables; i.e., "cold stimuli") and group membership and/or sex. I have tried running a series of repeated-measures ANOVAs separately for each of the different groupings of pain perception variables (i.e., heat stimuli threshold tests; cold stimuli threshold tests; heat pain magnitude estimates; cold pain magnitude estimates); however this solution does not feel optimal or adequate. My specific questions are:

1) is it appropriate to analyze data elicited from related (but not identical) questions/stimuli using repeated measures?

2) Is it appropriate to analyze chunks of the data (e.g., cold pain magnitude estimates; cold stimuli threshold tests) separately?

2) Would a different strategy (such as multilevel analysis / profile analysis / or some type of multivariate repeated measures ANOVA) be more appropriate?

3) General recommendations and/or best practices for analyzing data such as this.

Thank you for your feedback and input

Patrick Welch

note: I already searched the site for related questions (i.e., ANOVA with non-independent observations ; Parametric techniques for n-related samples) but believe the current question to be different enough to warrant unique consideration.

Answer 1

If temperature levels X1...X5 are specific degree values, I'm not sure how temperature and stimulus (hot/cold) can be completely crossed. I presume then that "temperature" consists of 5 ordinal categories, ranging from "likely to cause minimal discomfort" to "likely to cause maximum discomfort permissible by my research ethics board", which then permits crossing with hot/cold direction.

If this is the case, you are correct that the 3 response variables might not be expected to be independent, so you might increase your power (relative to 3 seperate analyses) if you do a multivariate test. Frankly, I must admit that my first inclination would be to simply do three separate analyses, one for each response variable, but that's because I'm not very knowledgeable with regards to the mechanics of multivariate tests. That said, given the likely dependence between the 3 response variables, significant results across 3 separate anovas might be most reasonably considered as manifestations of this dependence rather than 3 truly independent sets of phenomena.

I am pretty sure that you shouldn't lump all 3 response variables into a single univariate analysis by adding "response variable type" as a predictor variable; this approach could run into trouble if the scales and variances of your response variables are very different. Presumably, a multivariate analysis has features that take such differences into account. (However, I wonder if a mixed effects analysis might also be able to handle such differences? I'm new to mixed effects, but suspect I it might...)

Answer 2

As to your suggestion of using Multi-level models in this and the other thread, I see no benefit of approaching your analysis in this manner over repeated measures ANOVA. MLM are simply an extension of OLS regression, and offer an explicit framework to model group level (often referred to as "contextual") effects on lower level estimates. I would guess from your example you have no specific "contexts" besides that measures are repeated within individuals, and this is accounted for within the repeated ANOVA design (and your not interested in measuring effects of specific individuals anyway, only to control for this non-independence). Neither groups nor gender with what your description says are "contexts" in this sense, they can only be direct effects (or at least you can only observe if they have direct effects).

MLM just complicates things in this example IMO. It doesn't directly solve your problem that several dependent measures are non-independent, you can't measure any group level characteristic on your outcome because you only have two groups, and your hierarchy is cross classified and has three levels, (an observation is nested within only 1 individual, but the gender and group nestings are not mutually exclusive). All of the things you listed as purposes of the project can be accomplished using repeated ANOVA simply including group, gender, and group*gender interaction effects into the models.

It is beyond the scope of your question, but I would never agree that gender is a group observations are nested within.

Answer 3

I don't think that repeated measures is appropriate here - as I understand your protocol, the five temperatures (either hot or cold) are not repeated measures as they occur at different temperatures.

However, it does feel like you need some data reduction across the five levels in each temperature group. You might be well-served to consider some exploratory analysis to decide what the general shape of the relationship of pain perception to ordinal temperature input is across the hot and cold groups; then reduce that to a linear slope or exponential function; then model that single variable as your dependent variable with gender and A/B as factors.

Latent growth curve modeling might also be appropriate, but that is beyond me.