Dimensionality reduction refers to techniques for reducing many variables into a smaller number while keeping as much information as possible. One prominent method is [tag pca]

learn more… | top users | synonyms

0
votes
0answers
17 views

Why is feature normalization important in PCA? [duplicate]

If feature normalization is not performed, does the algorithm give incorrect results or is it it inefficient or both?
0
votes
0answers
13 views

Forward sequential feature selection improving classifier performance?

I was in a bit of a conversation with a co-worker about using forward selection. My training data is on order of ~6,000 w/ dimensionality of 1,200, and testing data on order of ~3,000. Currently, I'm ...
2
votes
1answer
51 views

Manifold learning: does an embedding function need to be well behaving?

I am trying to learn about manifold learning techniques; a family of methods in machine learning. According to this idea, there is a low ($d$) dimensional, hidden space where the real data generation ...
3
votes
1answer
77 views

Two broad categories of dimensionality reduction

As a starter in dimensionality reduction (DR), I recently acquired the following understanding. There are two very broad categories of DR techniques: We can compute an analytic form of mapping from ...
0
votes
0answers
26 views

k-NN classifier with a data-dependent distant measure?

As we know, in a $k$-NN classifier, we have to define a distance measure. Imagine a case where I use a certain dimensionality reduction technique to project my high-dimensional data to 2D, and then I ...
0
votes
0answers
16 views

What is the heuristic to decide number of components for LDA dimensionality reduction?

In the PCA case, I prefer to plot the variance and choose number of components regarding that plot's breaking point. In the LDA (linear disriminant analysis) case, what can be used for such an ...
0
votes
0answers
19 views

How to determine which variables need to be trimmed in PCA or Factor analysis?

Background: I'm working with log returns for about 400 tech stocks. I want to use PCA to reduce these into principal components (Internet companies, software developers, circuit board manufacturers, ...
8
votes
4answers
170 views

How can top $k$ principal components retain the predictive power on a dependent variable?

Suppose I am running a regression $Y \sim X$. Why by selecting top $k$ principle components of $X$, does the model retain its predictive power on $Y$? I understand that from ...
1
vote
0answers
23 views

Standardizing dimension reduction output

I understand that data is (typically) standardized (i.e. zero mean and unit variance) before dimension reduction technique such as PCA/LDA is applied. In addition to this, would it ever make sense to ...
1
vote
0answers
27 views

What is best practise for dimensionality reduction in rows of data

I was wondering what was best practise for dimensionality reduction in observations (as opposed to features) in a data-set? I often have data comprising of a multiple, random number of observations ...
2
votes
1answer
32 views

Can I apply factor analysis on multiple choice questions?

I am looking to validate a questionnaire and would like to know if I can use factor analysis on the multiple-choice questions (MCQ). Also, I have another section where I am asking about participants ...
3
votes
1answer
89 views

When should I use feature selection and when should I use dimensionality reduction techniques?

When should I use feature selection and dimensionality reduction? I know that feature selection is different from dimensionality reduction. But I don't know under what circumstances should I use ...
0
votes
1answer
46 views

Model Selection and RFE using caret

I'm faced with a high dimensional (samples=148, features=20000), supervised binary classification problem. Which I would like to approach with an ensemble of classifiers, that will classify using a ...
0
votes
0answers
42 views

In PCA, can the values in the principle component vectors which are close to zero be removed to see the important features? [duplicate]

In PCA, when I extract the principle component vectors, I am choosing the first vector with the largest corresponding eigenvalue. I notice that some of the values in this vector are close to zero. Can ...
1
vote
0answers
42 views

Dimension reduction for discrete qualitative and aggregated variables

I know about PCA for multiple dimensions of continuous features but here is a problem I have some trouble to find a method for. I don't have a list of individual countries but rather a discrete ...
2
votes
1answer
48 views

Support Vector Machines and the curse of dimensionality

I am reading this paper: "Automated MR image classification in temporal lobe epilepsy", by Focke et al. NeuroImage, 2012. The authors use support vector machines to classify subjects between healthy ...
0
votes
0answers
30 views

Are there any versions of t-SNE for streaming data?

My understanding of t-SNE and the Barnes-Hut approximation is that all data points are required so that all force interactions can be calculated at the same time and each point can be adjusted in the ...
0
votes
0answers
28 views

In R, is there an exact method for the political compass test?

In R, I am looking, in an exact way, for the method (or packages) used on the political compass test (see politicalcompass.org in the link Take the test) In my case, I have one data set consisting in ...
3
votes
1answer
73 views

What is the difference between feature selection and dimensionality reduction?

I know that both feature selection and dimensionality reduction aim towards reducing the number of features in the original set of features. What is the exact difference between the two if we are ...
0
votes
0answers
20 views

PCA reduction and low-reliability components

I'm working on a survey with 288 observation in total (108 complete answers used) and around 200 variables. I'm working on reducing those number using Principal Components Analysis, using R. Suppose ...
1
vote
0answers
66 views

How to reduce dimension of the sampling procedure?

I am stuck with this problem for a long time, hopefully I can get help here! Basically, I want to sample from a posterior distribution that looks like, \begin{align*} X &\sim ...
0
votes
1answer
36 views

LDA vs. SVM for Dimensionality Reduction

Whats the difference between LDA and Linear SVM for dimensionality reduction. I am little confuse as LDA also looks for projection that separates the classes of data and SVM we also look for ...
1
vote
0answers
32 views

Feature space reduction for tag prediction

[x-post] from stackoverflow. I am writing a ML module (python) to predict tags for a stackoverflow question (tag + body). My corpus is of around 5 million questions with title, body and tags for ...
0
votes
1answer
38 views

technical issues regarding to cluster analysis

Hi I would like to seek help with my cluster analysis using SAS. The main objective of the task is to segment customers into groups based on their similarity. The dataset contain mixed types of ...
1
vote
1answer
36 views

Intrinsic dimensionality estimation using Laplacian Eigenmaps

I learnt that I can look at the eigenspectrum of the kernel matrices computed by nonlinear spectral techniques in order to estimate the intrinsic dimensionality of a data-set. I use drtoolbox (The ...
0
votes
1answer
45 views

pruning Neural Network

Since a feedforward NN with a logistic function as activation function is not linear, does it make sense to reduce variables first with principal components or discriminant analysis? Because ...
2
votes
1answer
31 views

Clustering Consumer data with over 100 variables and 50000 rows each

I am tasked with performing a clustering exercise for a consumer survey dataset with the hopes of finding distinct consumer segments. In the past, I've done it using a variety of techniques- ...
0
votes
0answers
15 views

Dealing with seasonality when doing dimensionality reduction

I want to perform dimensionality reduction (in particular, PCA) on a data set that is highly seasonal. One approach that I came across when researching this is "seasonal PCA", where you split your ...
17
votes
1answer
887 views

Relationship between SVD and PCA. How to use SVD to perform PCA?

Principal component analysis (PCA) is usually explained via an eigen-decomposition of the covariance matrix. However, it can also be performed via singular value decomposition (SVD) of the data matrix ...
3
votes
1answer
69 views

How does CCA find a low-dimensional common subspace?

According to Wikipedia, canonical correlation analysis (CCA) finds pairs of canonical variables. CCA has also been used in many cases as dimensionality reduction tool to find low-dimensional ...
3
votes
1answer
161 views

Combining several variables into one outcome score: How is it done in the machine learning community?

I have got 8 cognitive (continuous) behaviour variables and would like to combine them into a composite score. I would then like to find the best predictors of this outcome (from about 50 predictors). ...
4
votes
2answers
68 views

How many components to use in PCA in order to preserve a certain amount of variance?

I want to reduce the dimensionality of my data with PCA, until it preserves $\alpha = 0.99$ of the variance. How do I decide how many eigenvectors I should use? So I'm looking for a function ...
0
votes
0answers
34 views

Reduction of species variables in vegetative analysis

Edited following helpful feedback. I have vegetation species data for a number of grassland habitat sites, and am preparing to begin Exploratory Data Analysis. Data was collected in 100 quadrats over ...
5
votes
0answers
57 views

When is t-SNE misleading?

Quoting from one of the authors: t-Distributed Stochastic Neighbor Embedding (t-SNE) is a (prize-winning) technique for dimensionality reduction that is particularly well suited for the ...
3
votes
2answers
108 views

Dimensionality reduction technique similar to LDA when class labels are probabilistic

Given discrete class labels, say True and False, LDA (linear discriminant analysis) can be used to perform discriminant dimensionality reduction and attempt to find a subspace that best separates the ...
0
votes
0answers
50 views

Fourier vs ARIMA vs Factor analysis vs PCA?

Background I'm currently analysing a timeseries. My data consists of half hourly observations of a certain measurement. This data is human generated, and so we believe there will be daily, or weekly, ...
4
votes
1answer
122 views

How to choose a kernel for kernel PCA?

What are the ways to choose what kernel would result in good data separation in the final data output by kernel PCA (principal component analysis), and what are the ways to optimize parameters of the ...
3
votes
1answer
73 views

What exactly is the procedure to compute principal components in kernel PCA?

In kernel PCA (principal component analysis) you first choose a desired kernel, use it to find your $K$ matrix, center the feature space via the $K$ matrix, find its eigenvalues and eigenvectors, then ...
1
vote
0answers
19 views

Derived Scale for Exploratory Factor Analysis

Is it appropriate to use a 10 point Likert scale for EFA in the form of Yes/No followed by strength of the response varying from Very Low to Very High? The scores would be assigned as follows Yes/VL ...
9
votes
3answers
390 views

Is there any value in dimensionality reduction of a data set where all variables are approximately orthogonal?

Suppose I have an $N$-dimensional data set where the $N$ dimensions are roughly orthogonal (have correlation zero). Is there any utility in terms of: Visualization Representation (for classifier ...
2
votes
1answer
86 views

Dimensionality reduction when number of samples is much larger than number of features

I was wondering what happens when the number of samples is much larger (e.g. $\times 200\:000$ times more) than the number of features? Is there any recommended way of reducing the samples' ...
0
votes
0answers
34 views

Reducing the number of independent variables [duplicate]

I am working on a research project and I am trying to find one variable (execution time) as a function of a number of other variables/ metrics which were logged during the execution of a job. There ...
1
vote
2answers
107 views

How to perform PCA in Matlab when number of dimensions is larger than number of observations?

I have a data matrix of say, $3000 \times 200$, i.e. I have $3000$-dimensional observations from $200$ subjects. How can I reduce the dimensionality to $1000$ in MATLAB? With bigger numbers, ...
1
vote
1answer
50 views

How to verify implementation of SVD in Javascript

I have implemented the SVD algortihm for my Node.js project for collaborative filtering of a sparse dataset based on this paper by GroupLens. For calculating the SVD, I am using the package node-svd ...
1
vote
2answers
114 views

How to build a predictive model with a billion of sparse features?

I am making a model to learn a dataset which has a big feature number and sparse samples (I am planning to use logistic regression). The feature number can be as big as 1,000,000,000. It is sparse ...
0
votes
0answers
25 views

Alternative methods of data reduction

There is obviously a lot of discussion on this site on appropriate data reduction techniques, particularly between factor analysis and principal component analysis. Soon, I will need to begin ...
1
vote
1answer
62 views

Is it possible to project a new vector onto the PC space using kernel PCA?

Let $X_{N \times d}$ be the data matrix, where $N$ is the number of samples and $d$ the size of the features space. Using kernel PCA (kPCA), one first computes a kernel matrix $K_{N \times N}$, and ...
0
votes
0answers
26 views

Using ANOVA to reduce number of variables

I am interested in using a machine learning technique (kriging) to model a function, based on a large number of variables. The large number of variables are causing overfitting problems, and I would ...
7
votes
2answers
140 views

A person repeatedly selects the two most similar items out of three. How to model/estimate a perceptual distance between the items?

A person is given three items, say pictures of faces, and is asked to pick out which two of the three faces are the most similar. This is repeated a large number of times with different combinations ...
10
votes
2answers
602 views

How to understand “nonlinear” as in “nonlinear dimensionality reduction”?

I am trying to understand the differences between the linear dimensionality reduction methods (e.g., PCA) and the nonlinear ones (e.g., Isomap). I cannot quite understand what the (non)linearity ...