Dimensionality reduction refers to techniques for reducing many variables into a smaller number while keeping as much information as possible. One prominent method is [tag pca]

learn more… | top users | synonyms

0
votes
0answers
13 views

technical issues regarding to cluster analysis

Hi I would like to seek help with my cluster analysis using SAS. The main objective of the task is to segment customers into groups based on their similarity. The dataset contain mixed types of ...
1
vote
1answer
19 views

Intrinsic dimensionality estimation using Laplacian Eigenmaps

I learnt that I can look at the eigenspectrum of the kernel matrices computed by nonlinear spectral techniques in order to estimate the intrinsic dimensionality of a data-set. I use drtoolbox (The ...
0
votes
1answer
41 views

pruning Neural Network

Since a feedforward NN with a logistic function as activation function is not linear, does it make sense to reduce variables first with principal components or discriminant analysis? Because ...
2
votes
1answer
24 views

Clustering Consumer data with over 100 variables and 50000 rows each

I am tasked with performing a clustering exercise for a consumer survey dataset with the hopes of finding distinct consumer segments. In the past, I've done it using a variety of techniques- ...
0
votes
0answers
14 views

Dealing with seasonality when doing dimensionality reduction

I want to perform dimensionality reduction (in particular, PCA) on a data set that is highly seasonal. One approach that I came across when researching this is "seasonal PCA", where you split your ...
15
votes
1answer
234 views

Relationship between SVD and PCA. How to use SVD to perform PCA?

Principal component analysis (PCA) is usually explained via an eigen-decomposition of the covariance matrix. However, it can also be performed via singular value decomposition (SVD) of the data matrix ...
3
votes
1answer
46 views

How does CCA find a low-dimensional common subspace?

According to Wikipedia, canonical correlation analysis (CCA) finds pairs of canonical variables. CCA has also been used in many cases as dimensionality reduction tool to find low-dimensional ...
3
votes
1answer
122 views

Combining several variables into one outcome score: How is it done in the machine learning community?

I have got 8 cognitive (continuous) behaviour variables and would like to combine them into a composite score. I would then like to find the best predictors of this outcome (from about 50 predictors). ...
4
votes
2answers
54 views

How many components to use in PCA in order to preserve a certain amount of variance?

I want to reduce the dimensionality of my data with PCA, until it preserves $\alpha = 0.99$ of the variance. How do I decide how many eigenvectors I should use? So I'm looking for a function ...
0
votes
0answers
24 views

Reduction of species variables in vegetative analysis

Edited following helpful feedback. I have vegetation species data for a number of grassland habitat sites, and am preparing to begin Exploratory Data Analysis. Data was collected in 100 quadrats over ...
5
votes
0answers
38 views

When is t-SNE misleading?

Quoting from one of the authors: t-Distributed Stochastic Neighbor Embedding (t-SNE) is a (prize-winning) technique for dimensionality reduction that is particularly well suited for the ...
3
votes
2answers
74 views

Dimensionality reduction technique similar to LDA when class labels are probabilistic

Given discrete class labels, say True and False, LDA (linear discriminant analysis) can be used to perform discriminant dimensionality reduction and attempt to find a subspace that best separates the ...
0
votes
0answers
40 views

Fourier vs ARIMA vs Factor analysis vs PCA?

Background I'm currently analysing a timeseries. My data consists of half hourly observations of a certain measurement. This data is human generated, and so we believe there will be daily, or weekly, ...
4
votes
1answer
74 views

How to choose a kernel for kernel PCA?

What are the ways to choose what kernel would result in good data separation in the final data output by kernel PCA (principal component analysis), and what are the ways to optimize parameters of the ...
3
votes
1answer
56 views

What exactly is the procedure to compute principal components in kernel PCA?

In kernel PCA (principal component analysis) you first choose a desired kernel, use it to find your $K$ matrix, center the feature space via the $K$ matrix, find its eigenvalues and eigenvectors, then ...
1
vote
0answers
15 views

Derived Scale for Exploratory Factor Analysis

Is it appropriate to use a 10 point Likert scale for EFA in the form of Yes/No followed by strength of the response varying from Very Low to Very High? The scores would be assigned as follows Yes/VL ...
9
votes
3answers
371 views

Is there any value in dimensionality reduction of a data set where all variables are approximately orthogonal?

Suppose I have an $N$-dimensional data set where the $N$ dimensions are roughly orthogonal (have correlation zero). Is there any utility in terms of: Visualization Representation (for classifier ...
2
votes
1answer
76 views

Dimensionality reduction when number of samples is much larger than number of features

I was wondering what happens when the number of samples is much larger (e.g. $\times 200\:000$ times more) than the number of features? Is there any recommended way of reducing the samples' ...
0
votes
0answers
34 views

Reducing the number of independent variables [duplicate]

I am working on a research project and I am trying to find one variable (execution time) as a function of a number of other variables/ metrics which were logged during the execution of a job. There ...
1
vote
2answers
72 views

How to perform PCA in Matlab when number of dimensions is larger than number of observations?

I have a data matrix of say, $3000 \times 200$, i.e. I have $3000$-dimensional observations from $200$ subjects. How can I reduce the dimensionality to $1000$ in MATLAB? With bigger numbers, ...
1
vote
1answer
36 views

How to verify implementation of SVD in Javascript

I have implemented the SVD algortihm for my Node.js project for collaborative filtering of a sparse dataset based on this paper by GroupLens. For calculating the SVD, I am using the package node-svd ...
1
vote
2answers
90 views

How to build a predictive model with a billion of sparse features?

I am making a model to learn a dataset which has a big feature number and sparse samples (I am planning to use logistic regression). The feature number can be as big as 1,000,000,000. It is sparse ...
0
votes
0answers
21 views

Alternative methods of data reduction

There is obviously a lot of discussion on this site on appropriate data reduction techniques, particularly between factor analysis and principal component analysis. Soon, I will need to begin ...
1
vote
1answer
52 views

Is it possible to project a new vector onto the PC space using kernel PCA?

Let $X_{N \times d}$ be the data matrix, where $N$ is the number of samples and $d$ the size of the features space. Using kernel PCA (kPCA), one first computes a kernel matrix $K_{N \times N}$, and ...
0
votes
0answers
19 views

Using ANOVA to reduce number of variables

I am interested in using a machine learning technique (kriging) to model a function, based on a large number of variables. The large number of variables are causing overfitting problems, and I would ...
7
votes
2answers
135 views

A person repeatedly selects the two most similar items out of three. How to model/estimate a perceptual distance between the items?

A person is given three items, say pictures of faces, and is asked to pick out which two of the three faces are the most similar. This is repeated a large number of times with different combinations ...
10
votes
2answers
583 views

How to understand “nonlinear” as in “nonlinear dimensionality reduction”?

I am trying to understand the differences between the linear dimensionality reduction methods (e.g., PCA) and the nonlinear ones (e.g., Isomap). I cannot quite understand what the (non)linearity ...
1
vote
0answers
29 views

Using SVD on features before SVM classification, when p >> N

So I am going through Hastie's Elements of Statistical computing, and in section 18.3.5 which deals with computational shortcuts when the number of dimensions $p$ is much larger that the number of ...
3
votes
1answer
80 views

Why are there only $N-1$ principal axes for $N$ data points if the number of dimensions is larger than $N$?

In PCA, when the number of dimensions $d$ is much much greater than the number of samples $N$, why is it that you will have at most $N-1$ non-zero eigenvectors? In other words, the rank of the ...
0
votes
0answers
13 views

Clustering cases on variables discovered in-sample via factor analysis?

My Data I have 2-hourly readings on approximately 10K sensors taken over the course of a year. The resulting time series look pretty similar day to day (though there are some longer term trends), and ...
5
votes
3answers
160 views

Why are mixed data a problem for euclidean-based clustering algorithms?

Most classical clustering and dimensionality reduction algorithms (hierarchical clustering, principal component analysis, k-means, self-organizing maps...) are designed specifically for numeric data, ...
1
vote
2answers
63 views

Motive behind preserving variance

Dimensionality reduction techniques preserve some properties of the data. I was wondering how preserving variance (as PCA does) can be helpful? Precisely speaking, PCA takes the covariance matrix and ...
6
votes
2answers
1k views

What is the difference between ZCA whitening and PCA whitening?

I am confused about ZCA whitening and normal whitening (which is obtained by dividing principal components by the square roots of PCA eigenvalues). As far as I know, $$\mathbf x_\mathrm{ZCAwhite} = ...
0
votes
0answers
23 views

Reduction of sparse features for machine learning

I'm trying to use a 1D histogram as a feature for machine learning. A histogram instance can be very sparse and the range of its bins is theoretically unbounded. Moreover it is expected that non-zero ...
0
votes
1answer
87 views

How can there be a linear correlation between two PCA components?

I perform principal component analysis (PCA) on a dataset, and then plot the first and the second principal components. I get the following phenomenon: one principal component appears to be a linear ...
0
votes
0answers
174 views

More recognizable Python implementation of Linear Discriminant Analysis?

I have been using scikit-learn's LDA implementation to do some experiments, and recently wanted to test out some modifications to the LDA derivation. I was looking at the Python implementation that ...
0
votes
0answers
10 views

Dimensionality vs Mulitvariability of time series objects

I am currently Clustering time series objects. Each individual object comprises 24 points in time and ca. 100 variables. So, it is represented by a 24x100 matrix. As I am not an English native ...
0
votes
1answer
32 views

Use fitted value from regression on subset of features as independent variable

I am working with a relatively large data set with 2K columns and many variables can be grouped together (a logistic regression). So I am thinking can I use fitted value from regression on subset of ...
2
votes
1answer
149 views

How to evaluate collinearity or correlation of predictors in logistic regression?

In linear regression it is possible to render predictors insignificant due to multicollinearity, as discussed in this question: How can a regression be significant yet all predictors be ...
1
vote
1answer
56 views

Algorithm to find subsets with high correlation

I have a reasonably large dataset (d) with predictor variables x1...xn and a target variable y. I can use recursive partitioning (such as CART or rpart in R) to find subsets of d with a high (or low) ...
4
votes
2answers
112 views

How to prove that the manifold assumption is correct?

In machine learning, it is often assumed that a data set lies on a smooth low-dimensional manifold (the manifold assumption), but is there any way to prove that assuming certain conditions are ...
2
votes
1answer
65 views

Is it valid to reduce dimensionality of the data with PCA before running nonlinear dimensionality reduction?

I have some very high dimensional data, and performing Locally Linear Embedding (LLE) is very time consuming. I also have to perform several LLEs, with varying parameters, to compute the optimal ...
1
vote
0answers
51 views

Order of preprocessing steps in a binary classification problem

I have these stages (ordered) for preprocessing in my binary classification problem. Dividing data based on criteria (class1 and class2 databases) Outlier ...
3
votes
1answer
196 views

Linear Discriminant Analysis and non-normal distributed data

If I understand correctly, a Linear Discriminant Analysis (LDA) assumes normal distributed data, independent features, and identical covariances for every class for the optimality criterion. Since ...
3
votes
1answer
95 views

Reference for dimension reduction techniques

This is a follow-up question to Is PCA appropriate for comparing subsets of panel data?. It turns out that, yes, PCA is appropriate. But there are also many other ways to reduce n-dimensional data to ...
7
votes
3answers
883 views

Why Python's scikit-learn LDA results are different from LDA in R or a step-by-step approach

I was using the Linear Discriminant Analysis (LDA) from the scikit-learn machine learning library (Python) for dimensionality reduction and was a little bit curious about the results. I am wondering ...
0
votes
0answers
46 views

Can PCA scores be used as dependent variable?

I am working on a research project where I have several questions from a survey data that measures the same underlying quantity (my dv), possibly each with some measurement error. I was thinking about ...
2
votes
2answers
65 views

Dimensionality reduction (PCA) for plotting text documents on a graph

I have 50 text documents There are 500 possible words, after a stop list has been applied My term/document sparse matrix is therefore 50x500 I'd like to cluster these documents. One easy way to do ...
2
votes
1answer
101 views

When is it appropriate to use PCA as a preprocessing step?

I understand that PCA is used for dimensionality reduction to be able to plot datasets in 2D or 3D. But I have also seen people applying PCA as a preprocessing step in classification scenarios where ...
3
votes
2answers
137 views

How to use SVD for dimensionality reduction to reduce the number of columns (features) of the data matrix? [duplicate]

My original data has many more columns (features) than rows (users). I am trying to reduce the features of my SVD (I need all of the rows). I found one method of doing so in a book called "Machine ...