2
$\begingroup$

If I want to compare the performance of PCA and PLS with nonlinear data, what do I have to measure ?

I know that I can use the MSE (but if I do not use a linear profile, I can't estimate the parameters of the linear regression, so how do I calculate it?

For the PLS, the data should follow a normal ditribution or it is an unsupervised method.

Which other methods would you propose to me for dimension reduction techniques?

Improve this question
$\endgroup$
1
  • $\begingroup$ Please specify some details about your problem, i.e. a) what actual problem you want to solve in the end? b) what are the types of variables? c) do you need supervised or unsupervised dimension reduction? Linear or nonlinear? If you answer this questions I will be able to help you! $\endgroup$
    – Paul
    Commented Jul 20, 2012 at 5:19

1 Answer 1

Reset to default
2
$\begingroup$

As you did not provide specific details on the problem I will give a short but comprehensive review of the dimension reduction techniques.

Generaly two problem statements considered:

Unsupervised dimension reduction.

We have a set of observations $S =\{X_i\}_{i=1}^{N}$ and need to find a reduced representation for it.

  • Linear models. Here PCA is a the most popular technique. MDS is another approach you can try.
  • Nonlinear models. Here we have a lot of different techiques: LTSA (I would suggest this in general), Isomap, LLE, and lots of their modifications.

All this methods (and a litlle bit more) are provided by special (yet free) MatLab toolbox.

Supervised dimension reduction (aka Feature Extraction).

We have a set of observations (both regressors and responses) $$S =\{X_i, Y_i\}_{i=1}^{N}, X_i\in\mathbb{R}^p, Y_i\in\mathbb{R}^q$$ and need to find a reduced representation for $X$s which preserves information about $Y$s. And we assume that the foolowing model holds: $$Y=f(X)+\varepsilon=g(\tilde{X}) +\varepsilon, \tilde{X}=XB, B\in\mathbb{R}^{p\times d}, d <p.$$ Verbally it means that we expect the regression $Y=f(X)$ to lye on some hyperplane of lower dimensionality. This problem statement considered in the case when we finally need a regression to be constructed.

  • Linear models. PLS is the most popular (and probably the only linear) technique.
  • Nonlinear models. MAVE, OPG, SAMM, SIR and some others are employed in this problem. Yocan find some MatLab implementations here.

In both problem statement dimensionalty selection is another individual problem. I can give more exhaustive comments on any of the subjects provided if you give more details on your problem and possibly choose the approach.

Improve this answer
$\endgroup$
6
  • $\begingroup$ I Have wrote a paper and i have only used (the combinition between PCA and wavelet analysis), the reviewer recommend to use some others approches and compare them with my method, i have sequence of profiles (taken for each unit of time) and each profile contains 1000 variables , also my profiles are nonlinear $\endgroup$
    – Sihem
    Commented Jul 20, 2012 at 22:53
  • $\begingroup$ i have used wavelet analysis (the reduce noise and compress data) after that PCA, it should be noted that if we take a look in the profile (at time t), the profile is approximately linear, but if we look to the variable nonlinear, i am interested to select some significant few variables $\endgroup$
    – Sihem
    Commented Jul 20, 2012 at 22:59
  • $\begingroup$ So the problem is just to find reduced-dimensional representation of n nonlinear profiles consisting of 1000 variables? And n << 1000? $\endgroup$
    – Paul
    Commented Jul 21, 2012 at 5:18
  • $\begingroup$ yes, the number of profiles is lower than the number of variables. and i want to retain only few significant variables and drawn each one of them vs (times= n profiles) $\endgroup$
    – Sihem
    Commented Jul 21, 2012 at 7:20
  • $\begingroup$ could you please provide an example plot of profile (if it is applicable) or actual data? And what is your final goal? $\endgroup$
    – Paul
    Commented Jul 21, 2012 at 7:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.