# Selecting features manually and proving the rest are redundant

I'm working with a gesture dataset, where each gesture has a variable number of frames, and each frame has the 3d position of 20 joints, so that each gesture is represented by a matrix of size frames x 60.

I know that some joints are redundant, since for example knowing the position of both shoulders pretty much determines the position of the chest and viceversa, at least for the poses in the gestures in my dataset.

Running PCA on the matrix of all the gestures stacked horizontally, I get that with just 30 dimensions I can retain 99% of the variance, but of course this is in the eigenvectors space.

How can I select a subset of the joints (equivalently, features) and prove that the rest are redundant, in a PCA sort of way? The simplest thing I could think of was to select some joints, use them as basis, project the frames onto the space they generate, and use the result as features, but a) the classification experiments I did with that didn't turn out well and b) I've no way of formally justifying the removal of features/joints with that approach.

• You could e.g. regress each features on all the rest, and see if there are any features that you can predict from others well enough (say with $R^2$ above some cutoff like 95%). If so, kick one of those out, and repeat. Commented Dec 23, 2014 at 23:20

## 1 Answer

I assume you mean "some features are redundant for classification", in that they do not contain andy class discriminatory information.

If you want to preserve class-discriminatory information while reducing dimension, you should use Linear Discriminant Analysis. PCA is not suited for this. Below is a copy/paste from Wikipedia.

LDA is closely related to principal component analysis (PCA) and factor analysis in that they both look for linear combinations of variables which best explain the data.[4] LDA explicitly attempts to model the difference between the classes of data. PCA on the other hand does not take into account any difference in class, and factor analysis builds the feature combinations based on differences rather than similarities. Discriminant analysis is also different from factor analysis in that it is not an interdependence technique: a distinction between independent variables and dependent variables (also called criterion variables) must be made.

LDA works when the measurements made on independent variables for each observation are continuous quantities. When dealing with categorical independent variables, the equivalent technique is discriminant correspondence analysis.[5][6]

• Zhubarb, I'm trying to determine which features are redundant, in the sense that I'm trying to find features that are roughly linear combinations of other features. In other words, I'm trying to find features that can be computed with little error by some function of the other features (preferrably, a linear function). As I understand it, LDA is basically a classifier; it tries to find categorical class labels as linear combinations of continuous features. I guess that what i'm looking for is more similar to Factor analysis or ANOVA. Commented May 7, 2014 at 2:27