# Regression without obvious dependent variable

Say I have 4 variables: Anxiety, Depression, Memory and Attention. I've given tests to subjects and now have a list of scores for each variable.

Say I want to know the relation between cognitive capacities (Memory and Attention) and mood (Depression and Anxiety). The problem here is that there is no way to select a theoretically supported dependent variable in my case. Bad cognitive faculties lead to worse mood, but bad mood also causes worse cognitive functioning.

So now I'm wondering: how do I model this thing? Should I just select the 2 mood variables as dependents and run Memory and Attention as predictors? If I reverse this does it matter for the results? Or does it only matter for the interpretation of the results?

I feel like I'm struggling with a very obvious problem but cannot figure out what to do. My only other thought was that regression simply does not apply to a model without direct effects and I'm doing this thing the whole wrong way.

• Sounds like either Principal Component Analysis or some form of partial least squares model might be getting toward what you want. – Glen_b Jun 30 '13 at 23:44
• Is this a problem that is addressed in Section 2.11, Normal Correlation Models in "Applied Linear Statistical Models" by Kutner el al? – Tom Jul 1 '13 at 15:56

Regression does imply a dependent variable.

But what exactly are you trying to do? Assuming all the variables are continuous, you can start with a correlation matrix. Where do go from there depends on why that isn't enough.

You should probably also plot all the variables, here you might start with a scatter plot matrix. Then, if you are using R, you could look at either trellis plots (with the lattice package) or faceting (with ggplot2).

• My purpose is wanting to know whether people with lower depression rates also display lower memory and/or attention rates. Same with anxiety; is there a significant link between anxiety and memory/attention scores? The purpose behind this is seeing if one of the two mood factors has a high relation with the cognitive domains. Say anxiety were to strongly be associated with memory but not attention then that would mean that people with high anxiety also suffer from worse memory but not necessarily attention. – user2487870 Jun 30 '13 at 23:32
• Also, I thought I could do a simple correlation matrix at first but the problem is that I have 10 cognitive domains to test, not two (I simplified this in the post to not overcomplicate things). If I were to run 20 individual correlations than this would seriously improve the chance to make a Type 1 error. – user2487870 Jun 30 '13 at 23:35
• user2487870 - those sound like things that you should explain more clearly in your question. – Glen_b Jun 30 '13 at 23:46
• If you have hypotheses about 10 domains, all of which may interact with one another, then you have a lot of things to test. The question of inflating and adjusting for type I error is not simple and has been discussed here before. My own view is that far too much concern is placed on this, as (at least in this type of research) you know in advance that the null isn't exactly true. ALL psychological traits are related to each other, the question is size of effect. But my view is not the only one! – Peter Flom Jun 30 '13 at 23:46

Since you have two matrices (memory- and attention related groups of variables), and want to know the relationship between them, I would suggest a PLS analysis (partial least squares).

PLS fits variables between two matrices and identifies these which have the largest explanatory power on the other set of data. One way of understanding it is that one set of variables contains the independent variables (X, memory), the other one contains the dependent variables (Y, anxiety).

However, one can also run PLS in a canonical mode in which the relationship is symmetrical and you do not assume that one set is dependent.

In R, a package implementing PLS / SPLS (sparse PLS) is called "mixOmics" and I had v. good experience using it.

As a complement to the other answers, which are very reasonable: your concern about "theory" when using regression where there are no "direct effects" seems to be a concern about causal interpretation, so I'll assume causality is the root issue.

First: you can always apply regression models without any causal interpretation because a regression model is just a model of (at least) the expectation of some conditional distribution, e.g. P(anxiety | depression, memory, attention).

Without a causal interpretation, one interprets a model parameter, say the coefficient on depression, roughly as follows: if we compare a group of people with some value of the depression measure to another group with a measure one unit higher we expect to see that the average anxiety score for that group are $\beta$ higher/lower than in the first.

Here the model is just a summary of the relationships you'd see if you plotted everything as @PeterFlom suggests.

The model itself doesn't change if you start to interpret it causally because the causal assumptions are about counterfactuals that require separate non-statistical assumptions. Although the relationship between statistical model specification and the identification of particular causal effects is a large one (and often conflated, as e.g. in the case of discussions of 'omitted variable bias'), the point here is that a regression model can be considered either as a summary of associations or as an attempt to identify a particular causal effect, as you choose.

Second: Again, roughly speaking, under a causal interpretation regressing anxiety against the other variables is consistent with (though does not uniquely imply) a causal story where those variables cause anxiety (directly or indirectly), no other variables that cause anxiety also have an effect on those variables, and the effect of the other variables on anxiety is the same for all subjects. But you do not have to sign up to this story if you don't want to interpret things causally.

It's also worth pointing out that the other answers here implicitly suggest other perhaps more defensible causal structures, e.g. @January suggests that there are two latent variables of which the four observed variables are only indicators i.e. effects, and that the causal or other relationship of interest holds between the latent variables rather than between the indicators. This leads to a more SEM type of approach and perhaps a more defensible causal story. It also leads to a different model.

I would also consider PLS as my first choice, as others suggested. Alternatively, I wonder if you could take each of these 4 variables in turn as your DV, with the other 3 as your IVs. Then you have 4 multivariate regression models. Similar to bivariate regression of x on y, or y on x, you could take some 'average' of these lines, but whether that would be an arithmetic, geometric, harmonic or other type is open for debate. Your result will then be an implicit, rather than an explicit equation in all 4 variates.