Dimensionality reduction refers to techniques for reducing many variables into a smaller number while keeping as much information as possible. One prominent method is [tag pca]

learn more… | top users | synonyms

0
votes
0answers
11 views

Using ANOVA to reduce number of variables

I am interested in using a machine learning technique (kriging) to model a function, based on a large number of variables. The large number of variables are causing overfitting problems, and I would ...
5
votes
1answer
90 views
+50

A person repeatedly selects the two most similar items out of three. How to model/estimate a perceptual distance between the items?

A person is given three items, say pictures of faces, and is asked to pick out which two of the three faces are the most similar. This is repeated a large number of times with different combinations ...
10
votes
2answers
545 views

How to understand “nonlinear” as in “nonlinear dimensionality reduction”?

I am trying to understand the differences between the linear dimensionality reduction methods (e.g., PCA) and the nonlinear ones (e.g., Isomap). I cannot quite understand what the (non)linearity ...
1
vote
0answers
20 views

Using SVD on features before SVM classification, when p >> N

So I am going through Hastie's Elements of Statistical computing, and in section 18.3.5 which deals with computational shortcuts when the number of dimensions $p$ is much larger that the number of ...
0
votes
0answers
17 views

Construct validity technique for small sample size

I have developed and pilot tested a quantitative survey instrument. I got only 40 respondents. No, I need to test its construct validity with the available data (N=40). What statistical analysis I can ...
2
votes
1answer
50 views

Why are there only $N-1$ principal axes for $N$ data points if the number of dimensions is larger than $N$?

In PCA, when the number of dimensions $d$ is much much greater than the number of samples $N$, why is it that you will have at most $N-1$ non-zero eigenvectors? In other words, the rank of the ...
0
votes
0answers
10 views

Clustering cases on variables discovered in-sample via factor analysis?

My Data I have 2-hourly readings on approximately 10K sensors taken over the course of a year. The resulting time series look pretty similar day to day (though there are some longer term trends), and ...
5
votes
3answers
121 views

Why are mixed data a problem for euclidean-based clustering algorithms?

Most classical clustering and dimensionality reduction algorithms (hierarchical clustering, principal component analysis, k-means, self-organizing maps...) are designed specifically for numeric data, ...
1
vote
2answers
60 views

Motive behind preserving variance

Dimensionality reduction techniques preserve some properties of the data. I was wondering how preserving variance (as PCA does) can be helpful? Precisely speaking, PCA takes the covariance matrix and ...
4
votes
2answers
298 views

What is the difference between ZCA whitening and PCA whitening?

I am confused about ZCA whitening and normal whitening (which is obtained by dividing principal components by the square roots of PCA eigenvalues). As far as I know, $$\mathbf x_\mathrm{ZCAwhite} = ...
0
votes
0answers
18 views

Reduction of sparse features for machine learning

I'm trying to use a 1D histogram as a feature for machine learning. A histogram instance can be very sparse and the range of its bins is theoretically unbounded. Moreover it is expected that non-zero ...
1
vote
1answer
80 views

How can there be a linear correlation between two PCA components?

I perform principal component analysis (PCA) on a dataset, and then plot the first and the second principal components. I get the following phenomenon: one principal component appears to be a linear ...
0
votes
0answers
105 views

More recognizable Python implementation of Linear Discriminant Analysis?

I have been using scikit-learn's LDA implementation to do some experiments, and recently wanted to test out some modifications to the LDA derivation. I was looking at the Python implementation that ...
0
votes
0answers
10 views

Dimensionality vs Mulitvariability of time series objects

I am currently Clustering time series objects. Each individual object comprises 24 points in time and ca. 100 variables. So, it is represented by a 24x100 matrix. As I am not an English native ...
0
votes
1answer
32 views

Use fitted value from regression on subset of features as independent variable

I am working with a relatively large data set with 2K columns and many variables can be grouped together (a logistic regression). So I am thinking can I use fitted value from regression on subset of ...
2
votes
1answer
109 views

How to evaluate collinearity or correlation of predictors in logistic regression?

In linear regression it is possible to render predictors insignificant due to multicollinearity, as discussed in this question: How can a regression be significant yet all predictors be ...
1
vote
1answer
52 views

Algorithm to find subsets with high correlation

I have a reasonably large dataset (d) with predictor variables x1...xn and a target variable y. I can use recursive partitioning (such as CART or rpart in R) to find subsets of d with a high (or low) ...
1
vote
0answers
38 views

Order of preprocessing steps in a binary classification problem

I have these stages (ordered) for preprocessing in my binary classification problem. Dividing data based on criteria (class1 and class2 databases) Outlier ...
2
votes
0answers
58 views

Using principal component analysis to reduce dimensions of data in R [closed]

I have a dataset which includes 4 separate measures of intelligence. To simplify my analysis, I wanted to express them as "g" a variable based on the shared variation of the 4 measures. A paper I read ...
2
votes
1answer
129 views

Linear Discriminant Analysis and non-normal distributed data

If I understand correctly, a Linear Discriminant Analysis (LDA) assumes normal distributed data, independent features, and identical covariances for every class for the optimality criterion. Since ...
3
votes
1answer
83 views

Reference for dimension reduction techniques

This is a follow-up question to Is PCA appropriate for comparing subsets of panel data?. It turns out that, yes, PCA is appropriate. But there are also many other ways to reduce n-dimensional data to ...
7
votes
3answers
624 views

Why Python's scikit-learn LDA results are different from LDA in R or a step-by-step approach

I was using the Linear Discriminant Analysis (LDA) from the scikit-learn machine learning library (Python) for dimensionality reduction and was a little bit curious about the results. I am wondering ...
0
votes
0answers
37 views

Can PCA scores be used as dependent variable?

I am working on a research project where I have several questions from a survey data that measures the same underlying quantity (my dv), possibly each with some measurement error. I was thinking about ...
2
votes
2answers
47 views

Dimensionality reduction (PCA) for plotting text documents on a graph

I have 50 text documents There are 500 possible words, after a stop list has been applied My term/document sparse matrix is therefore 50x500 I'd like to cluster these documents. One easy way to do ...
0
votes
0answers
22 views

Rotation of Mean Centred Variables in Principle Components Analysis

I'm looking to manually (Excel) perform PCA without any statistical packages such as R, but having trouble understanding how to rotate the original variables to find the maximum variance for the new ...
3
votes
1answer
85 views

When is it appropriate to use PCA as a preprocessing step?

I understand that PCA is used for dimensionality reduction to be able to plot datasets in 2D or 3D. But I have also seen people applying PCA as a preprocessing step in classification scenarios where ...
0
votes
1answer
54 views

How to use SVD for dimensionality reduction to reduce columns specifically?

My original data has many more columns (features) than rows (users). I'm trying to reduce the features of my SVD (I need all of the rows). I found one method of doing so in a book called "Machine ...
0
votes
1answer
34 views

Reconstructing a vector after projection

Suppose one has a matrix of data $X$, which is $n$ observations by $p$ dimensions. Let $P_\perp$ be a projection onto some $k<p$ dimensional subspace. Suppose one computes the principal direction ...
4
votes
2answers
256 views

Best practice for dimensionality reduction with PCA and LDA: does it make sense to combine them?

Assume I have a dataset for a supervised statistical classification task, e.g., via a Bayes' classifier. This dataset consists of 20 features and I want to boil it down to 2 features via ...
1
vote
0answers
54 views

Creating classification features from wavelet transformed time series

I'm interested in using a wavelet transform, Haar for example, to create classification variables from time series data to use in logistic regression. Simple example. Let's say I'm trying to predict ...
4
votes
2answers
103 views

How to reduce the dimension of $10^8$ vectors

I have $10^8$ vectors in $1000$ dimensions each. I would like to drastically reduce their dimensions. However PCA seems computationally infeasible. Are there near linear time methods to do ...
4
votes
1answer
188 views

How to distance and to MDS-plot objects according their complex shape

Suppose I have four basal forms of signal (blue, purple, red, green). I also have created transition forms between each other. If you carefully look on the picture below, you can see that for example ...
2
votes
1answer
49 views

Method to compare ratings from multiple different sources with missing data

I want a method to compare ratings from multiple sources and find a single measure that best reflects all the ratings. To give a specific example, let's call it "The fellowship review committee ...
0
votes
1answer
418 views

PCA on Binary Data

I having binary data set (yes/no), so can I apply PCA on that. Is it mathematically correct to do that. In my opinion Binary variable can only be subjected to logical operations, so how it can be ...
0
votes
0answers
17 views

Quality of NER classification decreases dramatically when sliding window transformation is performed

I'm writing a NER classificator now. That performs quite well even without window transformation (~80% F1-score for non-English language is quite well, AFAIK), but the strange thing is that when I ...
0
votes
1answer
82 views

PCA-based dimensionality reduction when the number of data points is less than the number of variables

Assume we are given a $p \times n$ (variables $\times$ data points) data matrix $X$, with $p > n$ (i.e. more variables than data points). Performing PCA on such a matrix yields an $n \times n$ ...
4
votes
2answers
264 views

Why does PCA maximize variance of the projection?

Christopher Bishop writes in his book (Pattern Recognition and Machine Learning) a proof, that each consecutive principal component maximizes the variance of the projection to one dimension, after the ...
0
votes
0answers
45 views

Machine learning from implied variables

I have a situation where we are detecting anomalies based on data implied from the table data. As an example, I have data on registered individuals spending time on the portal. Based on this, I have ...
1
vote
0answers
87 views

SVD application for a Boolean sparse Matrix

Basically, I am trying to have a recommender system based on SVD for a Boolean utility matrix. ie If at all some entries are present in the utility matrix, they will be 1 (I made it pseudo-implicit ...
2
votes
2answers
76 views

Which distribution has the larger variability? Is variance absolute, or relative to the mean?

I apologize in advance, I'm new to statistics. I have a large (millions) dataset (the US Census American Community Survey) with 286 attributes. I've calculated the mean, variance and standard ...
1
vote
1answer
62 views

Number of components in PCA

I believe I have a problem understanding PCA: I would like to use this technique to reduce the number of features of my problem. I originally have 10,000 features and 500 samples. However, the use of ...
1
vote
1answer
122 views

Principal component analysis with known uncertainty varying across both samples and variables

I want to do dimensionality reduction on a data set $X_{ij}$. In this case, $i$ indexes samples and $j$ indexes a large number of variables (densities at different locations in space). The units of ...
1
vote
1answer
101 views

PCA followed by LDA: dimension reduction strategy

I have a high dimensional dataset (n*p: 30 * 100) which I want to use as an testing dataset to build a two group classifier (LDA or QDA). I've read that you can do PCA to do an dimension reduction of ...
2
votes
2answers
169 views

Does fastICA require PCA to run at first?

I reviewed an application based paper saying that applying PCA before applying ICA (using fastICA package). My question is that does ICA (fastICA) requires PCA to run at first? This paper mentioned ...
1
vote
0answers
34 views

How do you explain dimension-reduction with statistics?

With statistics: how would you explain the dimensionality reduction and dimensionality addition? Like the conversion of a color picture to gray space so that a color blind person could more easily ...
7
votes
4answers
230 views

Non-orthogonal technique analogous to PCA

Suppose I have a 2D point dataset and I want to detect the directions of all the local maxima's of variance in the data, for example: PCA does not help in this situation as it is an orthogonal ...
2
votes
1answer
39 views

Selecting features manually and proving the rest are redundant

I'm working with a gesture dataset, where each gesture has a variable number of frames, and each frame has the 3d position of 20 joints, so that each gesture is represented by a matrix of size frames ...
0
votes
0answers
24 views

Subspace clustering with random transformation

One approach for clustering a high dimensional dataset is to use linear transformation, and the most common approaches are PCA and random projection (where random projection arises from the ...
0
votes
0answers
121 views

PCA Using prcomp in R

I'm trying to do principal component analysis (PCA) in R using the prcomp function. My input is a large matrix of 1,188 observations (rows) and 15,462 features (cols). I input this to the function ...
2
votes
0answers
54 views

Why linear transformation can improve classification accuracy when the dimensionality of data is high?

Let $X$ be an $m\times n$ ($m$: number of records, and $n$: number of attributes) dataset. When the number of attributes $n$ is large and the dataset $X$ is noisy, classification gets more ...