1
$\begingroup$

I am trying to create a multiple regression model in Python that takes hours slept, minutes of exercise, and my average daily mood to fit a 3D surface of $1^{st}$ (plane) to $5^{th}$ order polynomials. I have also calculated $R^2$ and adjusted $R^2$ values, which is all displayed on a figure like the one shown below.

enter image description here

My daily mood is measured by recording my mood approximately 2-3 hours after I wake up and 2-3 hours before I go to sleep and taking an average.

At first, I assumed the hours slept combined with my daily exercise would have an effect on my mood. However, while working on this project, I questioned if my rationale is backwards. What if my mood is independent and causes lack/excess sleep or motivation/pacification to exercise? For example: If I am having a bad day I may want to sleep longer and exercise less, and on a good day I may want to sleep only the minimum I need and go exercise. In that case, my model is backwards but it still shows realistic results (albeit poor).

Is there a way to determine this statistically? As in, is there a methodology to determine what variables (if any) are independent/dependent or is it intuition and understanding of the problem itself?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
0
$\begingroup$

You may rearrange your model multiple times to determine if one of the variables is best predicted by the others. But choosing your independent vs dependent variables should be chosen by intuition if possible, because context is extremely important.

For example, a regression model that predicts the price of a home based off of a home's square footage, location, and number of bathrooms is made solely for the purpose of predicting the price of a home. However, you could switch the dependent variable for an independent variable. This would be like predicting the home's square footage using the home's price and other variables. But, this model may not be useful to whoever is trying to predict home price. Changing the dependent variable may not be useful in every situation. The key is that the regression model will give you information about how well the independent variables you use predict the dependent variable. The model may also tell you about how important each independent variable is in predicting the dependent variable.

In contrast, if you are looking for the best possible regression in your data, make every variable a dependent variable in separate regression models. After creating the models, compare the R^2 values, p values and other relevant statistics to determine which variables best predict the data. In your example, this would mean making mood an independent variable and either sleep or exercise the dependent variable. For the example, lets make mood and exercise independent, sleep becomes dependent. If this new model is better than your current model with mood as the dependent variable, what this tells you is that exercise and mood are better at predicting your sleep rather than sleep and exercise are at predicting your mood. This may or may not be useful to you.

Note, do not confuse the mathematical definitions of independence and dependence in probability with the use of independent and dependent variables in this post. For two events A,B in a probability space, the two events are independent of each other if P(A|B) = P(A) and P(B|A) = P(B). Regression can even be used to determine how independent or dependent variables are, but the independent variables in regression are the variables used to predict the dependent variable. The dependent variable is the value being predicted by the model.

Improve this answer
$\endgroup$
5
  • $\begingroup$ It will be difficult to justify your recommendation to "compare ... p values and other relevant statistics to determine which variables best predict the data." This is at odds with what is known about regression. It's also difficult to determine what you really mean: exactly how does one compare models in which the roles of response and explanatory variables are permuted?? $\endgroup$
    – whuber ♦
    Commented Jul 5, 2022 at 19:55
  • $\begingroup$ Great comment whuber, I agree that my recommendation was vague. To be more clear, comparing the R^2 value, standard error, adjusted R^2 value, and p-values of the coefficients between the models with different dependent variables could tell you if one model is better. However, as you mentioned, it may not make sense to compare the two models because they predict different things. But, if one model is obviously better than the others based off of the statistics I mentioned above, than I believe that may answer Noah's question. $\endgroup$
    – Max Brown
    Commented Jul 5, 2022 at 20:21
  • 1
    $\begingroup$ I must strongly disagree: neither SEs, p-values, nor $R^2$ will tell you much of anything about whether one such model is "better" than another. (The SEs aren't even in the same kinds of units--hours, mood, and exercise--in the question.) What this question seems to be fishing for is some kind of dimensionality reduction: does there exist a low-dimensional manifold along which most of the observations appear to lie? Running these various univariate regressions will not be of much help in such an analysis. $\endgroup$
    – whuber ♦
    Commented Jul 5, 2022 at 20:25
  • $\begingroup$ Thank you, @whuber, I'm here to learn, do you have any resources to share about dimensionality reduction that could help me understand better? $\endgroup$
    – Max Brown
    Commented Jul 5, 2022 at 21:10
  • $\begingroup$ It's a huge and general subject. You can find a lot about it here on CV. $\endgroup$
    – whuber ♦
    Commented Jul 6, 2022 at 13:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.