0
$\begingroup$

enter image description here

Hi all,

Could anybody please help me?

I am trying to learn if there is any good approach in modern statistics that can fit this data well, without over-fitting.

Ok, here is another plot that I used "pch=".""... thanks for pointing that out to me!

enter image description here

Here is a 3D one: again, I want to be able to classify the {positive, negative}-ness of the z-axis...

enter image description here

Improve this question
$\endgroup$
9
  • 2
    $\begingroup$ If this is done in $R$, you should try replotting it with plot(...,pch="."). That is a quick-and-dirty way to see the density of the points better. $\endgroup$
    – cardinal
    Commented Feb 22, 2012 at 0:34
  • $\begingroup$ Can you say more about what you're trying to do? I suppose you're trying to use the variable on the $x$-axis to predict the variable on the $y$-axis but I'm not sure. $\endgroup$
    – Macro
    Commented Feb 22, 2012 at 0:51
  • $\begingroup$ yes, I am trying to do some predicting... especially I would like to figure out how to classify the folks on the y-axis into {positive, negative} classes... Thank you for asking! $\endgroup$
    – Luna
    Commented Feb 22, 2012 at 1:03
  • 1
    $\begingroup$ The folks on the $y$-axis? Each data point has both an $x$ and $y$ coordinate (and are apparently corresponding to individuals) so there's nobody on the $y$-axis who isn't also on the $x$-axis. What defines positive/negative? Absolute value of their $y$ measurement? $\endgroup$
    – Macro
    Commented Feb 22, 2012 at 1:04
  • $\begingroup$ Sorry for the confusion - I meant that I want to use the x values to predict/classify whether the y values will be positive or negative... This is a scatter plot, i.e. I plotted y vs. x ... The simplest one can be a linear regression, but I guess that's probably not the best way to do this ... I am also thinking of one more dimension/data-set, so that I can use x and y to predict/classify z. I guess before I move to high dimensional fitting, I'd better learn to do the above "simpler" things well. Thank you! $\endgroup$
    – Luna
    Commented Feb 22, 2012 at 1:08

3 Answers 3

Reset to default
4
$\begingroup$

Note that when you plot 2 independent but skewed variables against each other it can look like there is a relationship when there is not one (similar to your plots above). I would start by using the techniques outlined in:

 Buja, A., Cook, D. Hofmann, H., Lawrence, M. Lee, E.-K., Swayne,
 D.F and Wickham, H. (2009) Statistical Inference for exploratory
 data analysis and model diagnostics Phil. Trans. R. Soc. A 2009
 367, 4361-4383 doi: 10.1098/rsta.2009.0120

The vis.test function in the TeachingDemos package for R implements this test (or it is not that hard to do on you own using other tools).

If you cannot tell your data from permuted data using this method then any model you try to fit to the data will be overfitting. If there is enough of a relationship that you can see it here, then use the techniques others have mentioned (though with the skewness you may want to look at log transforms or other BoxCox transforms of your data).

Also, is every point in the above plot from a different subject? There seem to be some possible curves coming out of the main plot that look like they could be multiple measumerments over time from a single subject. If you have multiple measurements/points per subject with several subjects in the data, then that will also complicate any analysis, you will probably need to look at mixed effects models in that case.

Improve this answer
$\endgroup$
1
  • $\begingroup$ I spotted the same issue of the curves coming out of the main plot - if you follow the comments trail the OP confirmed that they are multiple measurements. $\endgroup$
    – Peter Ellis
    Commented Mar 23, 2012 at 21:20
0
$\begingroup$

You can start just by fitting a simple linear regression:

model <- lm( y~x, data=... )

Then you definitely wouldn't have to worry about overfitting :-). After this, you want to look into the poly() function. I doubt at this stage you would want to try very complicated equations. I'm not sure what else to suggest without knowing more information, such as if there is a known classification you would like to train with or if there are more variables available for fitting.

Improve this answer
$\endgroup$
1
  • $\begingroup$ maybe I should try SVM, by dividing this dataset into in-sample and out-sample? Or may I should try leave-K-out, etc.? $\endgroup$
    – Luna
    Commented Feb 22, 2012 at 2:51
0
$\begingroup$

If you just wont "to be able to classify the {positive, negative}-ness of the z-axis" within a probabilistic framework, then Gaussian Process Classification (GPC) may help you. SVM are ok, but you will find that SVM are sometimes just a special case of GPC. A useful reference manual is Gaussian Processes for Machine Learning, Carl Edward Rasmussen and Chris Williams, the MIT Press, 2006

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.