Regression without obvious dependent variable Say I have 4 variables: Anxiety, Depression, Memory and Attention. I've given tests to subjects and now have a list of scores for each variable.
Say I want to know the relation between cognitive capacities (Memory and Attention) and mood (Depression and Anxiety). The problem here is that there is no way to select a theoretically supported dependent variable in my case. Bad cognitive faculties lead to worse mood, but bad mood also causes worse cognitive functioning.
So now I'm wondering: how do I model this thing? Should I just select the 2 mood variables as dependents and run Memory and Attention as predictors? If I reverse this does it matter for the results? Or does it only matter for the interpretation of the results?
I feel like I'm struggling with a very obvious problem but cannot figure out what to do. My only other thought was that regression simply does not apply to a model without direct effects and I'm doing this thing the whole wrong way.
 A: Regression does imply a dependent variable.
But what exactly are you trying to do? Assuming all the variables are continuous, you can start with a correlation matrix. Where do go from there depends on why that isn't enough. 
You should probably also plot all the variables, here you might start with a scatter plot matrix.  Then, if you are using R, you could look at either trellis plots (with the lattice package) or faceting (with ggplot2). 
A: Since you have two matrices (memory- and attention related groups of variables), and want to know the relationship between them, I would suggest a PLS analysis (partial least squares).
PLS fits variables between two matrices and identifies these which have the largest explanatory power on the other set of data. One way of understanding it is that one set of variables contains the independent variables (X, memory), the other one contains the dependent variables (Y, anxiety).
However, one can also run PLS in a canonical mode in which the relationship is symmetrical and you do not assume that one set is dependent.
In R, a package implementing PLS / SPLS (sparse PLS) is called "mixOmics" and I had v. good experience using it.
A: As a complement to the other answers, which are very reasonable: your concern about "theory" when using regression where there are no "direct effects" seems to be a concern about causal interpretation, so I'll assume causality is the root issue.
First: you can always apply regression models without any causal interpretation because a regression model is just a model of (at least) the expectation of some conditional distribution, e.g. P(anxiety | depression, memory, attention).
Without a causal interpretation, one interprets a model parameter, say the coefficient on depression, roughly as follows: if we compare a group of people with some value of the depression measure to another group with a measure one unit higher we expect to see that the average anxiety score for that group are $\beta$ higher/lower than in the first.  
Here the model is just a summary of the relationships you'd see if you plotted everything as @PeterFlom suggests.
The model itself doesn't change if you start to interpret it causally because the causal assumptions are about counterfactuals that require separate non-statistical assumptions.  Although the relationship between statistical model specification and the identification of particular causal effects is a large one (and often conflated, as e.g. in the case of discussions of 'omitted variable bias'), the point here is that a regression model can be considered either as a summary of associations or as an attempt to identify a particular causal effect, as you choose.
Second: Again, roughly speaking, under a causal interpretation regressing anxiety against the other variables is consistent with (though does not uniquely imply) a causal story where those variables cause anxiety (directly or indirectly), no other variables that cause anxiety also have an effect on those variables, and the effect of the other variables on anxiety is the same for all subjects.  But you do not have to sign up to this story if you don't want to interpret things causally.
It's also worth pointing out that the other answers here implicitly suggest other perhaps more defensible causal structures, e.g. @January suggests that there are two latent variables of which the four observed variables are only indicators i.e. effects, and that the causal or other relationship of interest holds between the latent variables rather than between the indicators.  This leads to a more SEM type of approach and perhaps a more defensible causal story.  It also leads to a different model.
A: I would also consider PLS as my first choice, as others suggested. Alternatively, I wonder if you could take each of these 4 variables in turn as your DV, with the other 3 as your IVs. Then you have 4 multivariate regression models. Similar to bivariate regression of x on y, or y on x, you could take some 'average' of these lines, but whether that would be an arithmetic, geometric, harmonic or other type is open for debate. Your result will then be an implicit, rather than an explicit equation in all 4 variates. 
