We are trying to predict values of variable A having N other variables. What we do, we calculate Pearson correlation between A and each of the other N variables, for last M values, using fixed M. We use variable with largest correlation coefficient as predictor.

This scheme works fine when analyzing N variables during 24 month and decreasing between month 25-36.
What could come as improvement of Pearson correlation in this context?

Variables are bi-valued.

  • 1
    $\begingroup$ Can you edit and add more context? It's hard for me to tell from your current description what you are doing and what your concerns are. $\endgroup$ – cardinal Apr 3 '11 at 20:21
  • $\begingroup$ Edited question. $\endgroup$ – Andrei Apr 3 '11 at 20:30
  • $\begingroup$ @cardinal: does this answer your question. $\endgroup$ – Andrei Apr 4 '11 at 7:09
  • $\begingroup$ Why use only one predictor variable? When you say "b-valued" do you mean dichotomous? $\endgroup$ – Michael Bishop Apr 4 '11 at 19:00
  • $\begingroup$ @Michael: zeros and ones $\endgroup$ – Andrei Apr 5 '11 at 18:53

This is generally hard to tell without knowing what the problem exactly is, but I would advise you to try some machine learning methods. For start you may try random forest, which is almost trivial to apply and quite probably will achieve better accuracy than just using one, best-correlated variable.
Also, it will produce importance measure which will tell you which variables contribute most to the prediction accuracy -- possibly taking into account even quite complex multivariate interactions obviously invisible to Pearson's correlation.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.