Dimensionality reduction refers to techniques for reducing many variables into a smaller number while keeping as much information as possible. One prominent method is [tag pca]

learn more… | top users | synonyms

4
votes
2answers
97 views

What is the difference between ZCA whitening and PCA whitening?

I am confused about ZCA whitening and normal whitening (which is obtained by dividing principal components by the square roots of PCA eigenvalues). As far as I know, $$\mathbf x_\mathrm{ZCAwhite} = ...
0
votes
0answers
16 views

Reduction of sparse features for machine learning

I'm trying to use a 1D histogram as a feature for machine learning. A histogram instance can be very sparse and the range of its bins is theoretically unbounded. Moreover it is expected that non-zero ...
1
vote
1answer
73 views

How can there be a linear correlation between two PCA components?

I perform principal component analysis (PCA) on a dataset, and then plot the first and the second principal components. I get the following phenomenon: one principal component appears to be a linear ...
0
votes
0answers
42 views

More recognizable Python implementation of Linear Discriminant Analysis?

I have been using scikit-learn's LDA implementation to do some experiments, and recently wanted to test out some modifications to the LDA derivation. I was looking at the Python implementation that ...
0
votes
0answers
10 views

Dimensionality vs Mulitvariability of time series objects

I am currently Clustering time series objects. Each individual object comprises 24 points in time and ca. 100 variables. So, it is represented by a 24x100 matrix. As I am not an English native ...
0
votes
1answer
29 views

Use fitted value from regression on subset of features as independent variable

I am working with a relatively large data set with 2K columns and many variables can be grouped together (a logistic regression). So I am thinking can I use fitted value from regression on subset of ...
2
votes
1answer
71 views

How to evaluate collinearity or correlation of predictors in logistic regression?

In linear regression it is possible to render predictors insignificant due to multicollinearity, as discussed in this question: How can a regression be significant yet all predictors be ...
1
vote
1answer
48 views

Algorithm to find subsets with high correlation

I have a reasonably large dataset (d) with predictor variables x1...xn and a target variable y. I can use recursive partitioning (such as CART or rpart in R) to find subsets of d with a high (or low) ...
1
vote
0answers
30 views

Order of preprocessing steps in a binary classification problem

I have these stages (ordered) for preprocessing in my binary classification problem. Dividing data based on criteria (class1 and class2 databases) Outlier ...
2
votes
0answers
52 views

Using principal component analysis to reduce dimensions of data in R [closed]

I have a dataset which includes 4 separate measures of intelligence. To simplify my analysis, I wanted to express them as "g" a variable based on the shared variation of the 4 measures. A paper I read ...
2
votes
1answer
78 views

Linear Discriminant Analysis and non-normal distributed data

If I understand correctly, a Linear Discriminant Analysis (LDA) assumes normal distributed data, independent features, and identical covariances for every class for the optimality criterion. Since ...
3
votes
1answer
68 views

Reference for dimension reduction techniques

This is a follow-up question to Is PCA appropriate for comparing subsets of panel data?. It turns out that, yes, PCA is appropriate. But there are also many other ways to reduce n-dimensional data to ...
7
votes
3answers
423 views

Why Python's scikit-learn LDA results are different from LDA in R or a step-by-step approach

I was using the Linear Discriminant Analysis (LDA) from the scikit-learn machine learning library (Python) for dimensionality reduction and was a little bit curious about the results. I am wondering ...
0
votes
0answers
35 views

Can PCA scores be used as dependent variable?

I am working on a research project where I have several questions from a survey data that measures the same underlying quantity (my dv), possibly each with some measurement error. I was thinking about ...
2
votes
2answers
42 views

Dimensionality reduction (PCA) for plotting text documents on a graph

I have 50 text documents There are 500 possible words, after a stop list has been applied My term/document sparse matrix is therefore 50x500 I'd like to cluster these documents. One easy way to do ...
0
votes
0answers
21 views

Rotation of Mean Centred Variables in Principle Components Analysis

I'm looking to manually (Excel) perform PCA without any statistical packages such as R, but having trouble understanding how to rotate the original variables to find the maximum variance for the new ...
3
votes
1answer
71 views

When is it appropriate to use PCA as a preprocessing step?

I understand that PCA is used for dimensionality reduction to be able to plot datasets in 2D or 3D. But I have also seen people applying PCA as a preprocessing step in classification scenarios where ...
0
votes
1answer
43 views

How to use SVD for dimensionality reduction to reduce columns specifically?

My original data has many more columns (features) than rows (users). I'm trying to reduce the features of my SVD (I need all of the rows). I found one method of doing so in a book called "Machine ...
0
votes
0answers
21 views

Reconstructing a vector after projection

Suppose one has a matrix of data $X$, which is $n$ observations by $p$ dimensions. Let $P_\perp$ be a projection onto some $k<p$ dimensional subspace. Suppose one computes the principal direction ...
3
votes
2answers
180 views

Best practice for dimensionality reduction using Principal Component Analysis (PCA) and/or Linear Discriminant Analysis (LDA)

Assume I have a dataset for a supervised statistical classification task, e.g., via a Bayes' classifier. This dataset consists of 20 features and I want to boil it down to 2 features via ...
1
vote
0answers
40 views

Creating classification features from wavelet transformed time series

I'm interested in using a wavelet transform, Haar for example, to create classification variables from time series data to use in logistic regression. Simple example. Let's say I'm trying to predict ...
4
votes
2answers
103 views

How to reduce the dimension of $10^8$ vectors

I have $10^8$ vectors in $1000$ dimensions each. I would like to drastically reduce their dimensions. However PCA seems computationally infeasible. Are there near linear time methods to do ...
4
votes
1answer
174 views

How to distance and to MDS-plot objects according their complex shape

Suppose I have four basal forms of signal (blue, purple, red, green). I also have created transition forms between each other. If you carefully look on the picture below, you can see that for example ...
2
votes
1answer
45 views

Method to compare ratings from multiple different sources with missing data

I want a method to compare ratings from multiple sources and find a single measure that best reflects all the ratings. To give a specific example, let's call it "The fellowship review committee ...
0
votes
1answer
246 views

PCA on Binary Data

I having binary data set (yes/no), so can I apply PCA on that. Is it mathematically correct to do that. In my opinion Binary variable can only be subjected to logical operations, so how it can be ...
0
votes
0answers
14 views

Quality of NER classification decreases dramatically when sliding window transformation is performed

I'm writing a NER classificator now. That performs quite well even without window transformation (~80% F1-score for non-English language is quite well, AFAIK), but the strange thing is that when I ...
0
votes
1answer
69 views

PCA-based dimensionality reduction when the number of data points is less than the number of variables

Assume we are given a $p \times n$ (variables $\times$ data points) data matrix $X$, with $p > n$ (i.e. more variables than data points). Performing PCA on such a matrix yields an $n \times n$ ...
4
votes
2answers
203 views

Why does PCA maximize variance of the projection?

Christopher Bishop writes in his book (Pattern Recognition and Machine Learning) a proof, that each consecutive principal component maximizes the variance of the projection to one dimension, after the ...
0
votes
0answers
43 views

Machine learning from implied variables

I have a situation where we are detecting anomalies based on data implied from the table data. As an example, I have data on registered individuals spending time on the portal. Based on this, I have ...
0
votes
0answers
81 views

SVD application for a Boolean sparse Matrix

Basically, I am trying to have a recommender system based on SVD for a Boolean utility matrix. ie If at all some entries are present in the utility matrix, they will be 1 (I made it pseudo-implicit ...
2
votes
2answers
74 views

Which distribution has the larger variability? Is variance absolute, or relative to the mean?

I apologize in advance, I'm new to statistics. I have a large (millions) dataset (the US Census American Community Survey) with 286 attributes. I've calculated the mean, variance and standard ...
1
vote
1answer
58 views

Number of components in PCA

I believe I have a problem understanding PCA: I would like to use this technique to reduce the number of features of my problem. I originally have 10,000 features and 500 samples. However, the use of ...
1
vote
1answer
116 views

Principal component analysis with known uncertainty varying across both samples and variables

I want to do dimensionality reduction on a data set $X_{ij}$. In this case, $i$ indexes samples and $j$ indexes a large number of variables (densities at different locations in space). The units of ...
1
vote
1answer
81 views

PCA followed by LDA: dimension reduction strategy

I have a high dimensional dataset (n*p: 30 * 100) which I want to use as an testing dataset to build a two group classifier (LDA or QDA). I've read that you can do PCA to do an dimension reduction of ...
2
votes
2answers
126 views

Does fastICA require PCA to run at first?

I reviewed an application based paper saying that applying PCA before applying ICA (using fastICA package). My question is that does ICA (fastICA) requires PCA to run at first? This paper mentioned ...
1
vote
0answers
33 views

How do you explain dimension-reduction with statistics?

With statistics: how would you explain the dimensionality reduction and dimensionality addition? Like the conversion of a color picture to gray space so that a color blind person could more easily ...
7
votes
4answers
213 views

Non-orthogonal technique analogous to PCA

Suppose I have a 2D point dataset and I want to detect the directions of all the local maxima's of variance in the data, for example: PCA does not help in this situation as it is an orthogonal ...
2
votes
1answer
37 views

Selecting features manually and proving the rest are redundant

I'm working with a gesture dataset, where each gesture has a variable number of frames, and each frame has the 3d position of 20 joints, so that each gesture is represented by a matrix of size frames ...
0
votes
0answers
21 views

Subspace clustering with random transformation

One approach for clustering a high dimensional dataset is to use linear transformation, and the most common approaches are PCA and random projection (where random projection arises from the ...
0
votes
0answers
104 views

PCA Using prcomp in R

I'm trying to do principal component analysis (PCA) in R using the prcomp function. My input is a large matrix of 1,188 observations (rows) and 15,462 features (cols). I input this to the function ...
2
votes
0answers
52 views

Why linear transformation can improve classification accuracy when the dimensionality of data is high?

Let $X$ be an $m\times n$ ($m$: number of records, and $n$: number of attributes) dataset. When the number of attributes $n$ is large and the dataset $X$ is noisy, classification gets more ...
0
votes
1answer
168 views

Result from Step by step PCA implementation differs from `matplotlib.mlab.PCA()`, would be nice if someone can help me finding the source

I was reading this nice article and tried to implement this step by step guide in Python, and then I compared the results using the Python function from the matplotlib library: ...
0
votes
1answer
29 views

Finding most significant data set within data

Basically, I'm conducting research based on two types of data: Noise levels and the temperature of a room. I've recoded data for 2 days.. I am using the spearman's correlation methods to determine ...
0
votes
0answers
63 views

Dimension reduction for likert questions, cronbach alpha

I need some input on how to proceed with my data. I have collected data from household survey on 180 sample to find out the importance given to the attributes in a residential location decision. The ...
3
votes
1answer
169 views

Machine learning techniques for spam detection, and in general for text classification

I am going to configure a system for spam detection. What I have is a dataset of labeled (spam/not-spam) strings containing, mostly, sentences. I have a background in machine learning techniques, but ...
1
vote
0answers
75 views

Comparing isomap residual variance to pca variance

I am using R princomp function (from stats package) to run a PCA on a data set and I want to compare its output to that of the nonlinear dimensionality reduction method ISOMAP, which I am using under ...
1
vote
0answers
42 views

Define a hypersherical neighborhood around a target point with radius r'

This is a homework problem that I need some help with. QUESTION: We assume that N data points are uniformly distributed in a 100-dimensional unit hypersphere (i.e. $r = 1$) centered at the ...
1
vote
1answer
94 views

Linear model predictor selection. Which method to use ?

From what I understand, there are 3 main types of predictor selection method for linear models, namely, 1 Subset Selection, 2 Shrinkage and 3 Dimension Reduction. The subset selection includes the ...
0
votes
1answer
43 views

Combining variables

I'd like to combine several variables into one variable. Here is some context: Let's say I have two variables Red.Beads and ...
0
votes
1answer
38 views

When accuracy in the cross-validation process less,is reducing the features a good idea?

I am doing a project for classifying the presence of cars/bikes in an image.I have extracted the features from the images(data-set of cars and images not belonging to that of cars) and applied K-means ...