Skip to main content
2 of 2
added 22 characters in body
Galen
  • 9.7k
  • 3
  • 27
  • 61

I agree with Peter Flom's answer and Stephan Kolassa's answer to a substantial extent. Like Stephan, I think a highly-limited analysis could be performed, but I want to give my two cents on the approach I would take.

Similarly to Jarle Tufto's comment, I would take an ordinal regression approach.

Further, I would go with mixed effects on my parameters with the random effects being for the two surveys/groups. This approach at least will give better statistical precision for the fixed effects.

I would follow an exploratory Bayesian workflow that explores different models iteratively and thoughtfully. Although fanciful and hopeful in your case, it might allow you to find provisional evidence for latent structures, such as latent mixtures of parameters. I wouldn't hold my breath, but finding such a thing in an exploratory analysis might suggest future directions from a more confirmatory approach.

I recommend studying McElreath's lecture Statistical Rethinking 2023 - 11 - Ordered Categories if you plan on pursuing such an analysis. Not only does he walk through a Bayesian approach to fitting an ordinal regression on survey data, but you can also see his emphasis on causal assumptions.

Developing a causal model for your system of study will help you understand, analyze, and explain the flaws in your statistical design. I expect that you are conditioning on a collider, and I am doubtful that sensitivity analysis to confounds will help you.

Galen
  • 9.7k
  • 3
  • 27
  • 61