0
$\begingroup$

I am working on a prediction problem. I have a programming background and a small statistics one. If I have two dependent variables and one independent one, how do I work out if the two are co-related?

For example, x1 has a correlation coefficient of .6 and x2 has one of .4. I hypothesize that x1 and x2 are not totally dependent (identity) and not totally independent to each other. Is there a way to analyze this situation mathematically to eventually decide weights of a final regression equation?

I suppose I could put one x into bins and make a piecewise equation based on certain cutoffs. But, this seems not to be optimal and in reality there are many x's. I have not found any good literature on how to differentiate weights of dependent variables. I know machine learning is designed for this but I would like a 'old school slide ruler' approach where I can understand the differences of variance.

I am in a exploratory phase where given two (and really many more) variables figure out the outcome. If the two variables are co-related then some of the variance is overlapping. I would like to know how much overlaps and how to weight the variables in an ending regression equation. (this my not be the right terminology so please bear with me.)

Improve this question
$\endgroup$
2
  • $\begingroup$ Which two are you attempting to correlate (and why not just calculate that correlation)? Are those correlations with the independent variable? (Please edit instead of just replying.) $\endgroup$
    – Nick Stauner
    Jul 3, 2014 at 19:53
  • $\begingroup$ I am in a exploratory phase where given two (and really many more) variables figure out the outcome. If the two variables are co-related then some of the variance is overlapping. I would like to know how much overlaps and how to weight the variables in an ending regression equation. (this my not be the right terminology so please bear with me.) $\endgroup$
    – Nate C
    Jul 3, 2014 at 19:58

1 Answer 1

Reset to default
2
$\begingroup$

I don't want to sound rude here but I think you'd be better off getting a firm background in Intro. Statistics before you start trying to implement any of the techniques of machine learning. Not only will this answer your question but also it will teach you the basic terminology, which will help you word your questions in the future.

But, to clarify a few things that you may have mistyped in your question (as it reads currently, at least) and also hopefully point you in the right direction:

  • The X's are the Independent variables (also called the "Explanatory variables"), the "y" that you're trying to "explain" is the Dependent variable.
  • If you want to know if two Independent variables are correlated, the easiest thing to do is just to calculate their Correlation. [Note: Even if they aren't totally independent you can often still run your regression without any problems.]
  • It's the regular, run-of-the-mill regression (OLS) that chooses the "weights" for you, in effect (in the form of estimated slope coefficients). [Note: If you're still fairly new to Statistics then don't bother with anything more complicated than OLS (such as Generalized Least Squares)].
  • The issue of Multicollinearity may be what you're referring to in your question. To explain it imprecisely, Multicollinearity is when 2 or more X's are correlated enough to screw up your OLS regression. Basically, if you run your regression and you end up with a high R^2 but none of the t-values of the individual coefficients are statistically significant, then you probably have Multicollinearity.
  • Also, it sounds like you're looking to calculate a run-of-the-mill ANOVA table--have you tried that yet?

Hopefully some of this will be helpful to you. I can add more pointers if it is--I going to wait for some feedback from you before I continued, however. If these bullet points are totally off then I apologize. Also, it would help if you continue to edit your post accordingly since it's kind of hard to parse as it is (for me, at least).

Good luck!

-- Last thing: What are you using to calculate all this? (e.g. Excel, R, TI-83)

Improve this answer
$\endgroup$
2
  • $\begingroup$ Perfect Answer. Multicollinearity is the search term I wanted. OLS is new to me and I will search that also. That sounds about the right approach Thank you so much for taking the time to give the right 'words' to research. I'm am using sample data with a postgresql relation database. In other words, I roll all of my own data and need a complete understanding of it. $\endgroup$
    – Nate C
    Jul 3, 2014 at 21:36
  • $\begingroup$ Hey, thanks for the feedback--I was worried that it might come off as sounding condescending and would totally miss the mark. Oh, by the way, an OLS regression is basically just your run-of-the-mill regression (i.e. it's the technical name for the regression you run in Stats 101). Also, you might find Think Stats helpful considering your programming background (a free pdf is available via that link). $\endgroup$
    – Steve S
    Jul 3, 2014 at 21:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.