Refers to techniques for reducing a large number of variables to a smaller number while preserving as much information as possible. Prominent methods include PCA, MDS, Isomap, etc.

learn more… | top users | synonyms

0
votes
0answers
29 views

PCA Transforming back to original space from a subset of principal components [on hold]

I applied PCA on a 12000 x 500 data set (12000 data points with 500 features). PCA gave me 12000 x 20 data (20 features). Is there any way to transform the results back to the original data space? (To ...
2
votes
1answer
49 views

Meaning of “reconstruction error” in PCA and LDA

I am implementing PCA, LDA, and Naive Bayes, for compression and classification respectively (implementing both an LDA for compression and classification). I have the code written and everything ...
1
vote
1answer
60 views

Can PCA be applied twice or more?

I have a very high-dimensional dataset with 27k features. I want to a reduce the dimensionality of the dataset. I want to use PCA to reduce the dimensionality to 2 as the toolbox I am using expects 2 ...
0
votes
0answers
24 views

Units of measurement on a two-dimensional PCA scatterplot

I have a dataset that consists of the average consumption of 200~ types of food in grams per person per week for several countries, much like the dataset described at the bottom of this page Eating in ...
0
votes
1answer
49 views

data normalization after dimension reduction for classification

The classifier is KNN or RBF-SVM. After doing dimension reduction (e.g., PCA, LDA or KPCA, KLDA), does it need to do normalization before classification? In LIBSVM ...
0
votes
0answers
23 views

Inferences from PCA plot

I have done a dimensionality reduction of binary labelled data (0,1 labels) from 300 features to 2 features. The plot looks like - What kind of inferences can I make from this plot? Can I infer - ...
0
votes
0answers
32 views

How to compare PCA with KPCA for dimension reduction?

Both linear principal component analysis (PCA) and kernel principal component analysis (KPCA) are unsupervised dimension reduction methods. I have a dataset with $4000$ training samples and $40000$ ...
0
votes
0answers
16 views

t-SNE: Aren't identical words meant to stack in the same place?

I looked at t-SNE examples on Laurens van der Maatens' page and noticed, that some words/phrases appear multiple times in slightly different locations (for example zoom in at t-SNE example, middle to ...
0
votes
1answer
97 views

How does Principal Component Analysis help me understand my data?

I have a dataset which contains 10000 examples. Each example has 100 dimensions. These dimensions have the same scale. I clustered all examples using their 100-dimensional vectors and drew the elbow ...
0
votes
0answers
16 views

Tunable unsupervised dimension reduction?

I am trying to do some dimension reduction as the pre-processing for nearest neighbor search. I have tried PCA and it seems to be OK. What I don't like PCA is that it has no tuning parameter. I hope ...
0
votes
0answers
11 views

Using clusters in parameters to divide up a logistic regression?

I have a data set that is a bit involved, where I try to find the parameters that are the most helpful when I try to predict a binary response (the behaviour of human raters). The number of possible ...
0
votes
0answers
25 views

Dimensionality reduction for a mapping

I have a dataset that I measured in 100 dimensions (i.e. each sample contains 100 values). I want to dimensionality reduce the data to 10 dimensions (i.e find a mapping where 10 values per sample can ...
0
votes
1answer
18 views

Groups of highly correlated variables within Regressions - dimension reduction by variable groups

Dear Cross Validated Users, I am addressing myself to you with a question for which I couldn't find an answer despite intensive googling. I want to run a regression with a multitude of independent ...
0
votes
1answer
28 views

what is the best approach for factor analysis when the data has more attributes than inputs? [closed]

i think i should ask the question like this: i am having a data set of 20 participants with 89 attributes , almost all of the attributes have values between 5 to 0 and there exist more than 20 ...
1
vote
0answers
36 views

Train & predict probabilities using LDA having multiple collinearities

I am trying to fit an LDA model and predict conditional probabilities of class membership with it. I believe I understand the basic method to do this using the covariance matrix and class means, but ...
3
votes
1answer
35 views

Under what conditions would clustering on top of Principal Components would return different result (and worse) than clustering on the data itself?

Since Principal components capture most of the information, clustering on them should provide similar result as that of the clustering on the original data. As such, it seems to me (who's not a ...
0
votes
1answer
53 views

Is that correct about dimensionality reduction and clustering?

Could you please help me to understand it because I'm not sure if I got it correctly. Let's say I have a dataset, of persons, with 100 features, various characteristics like height, weight, age, etc. ...
0
votes
0answers
33 views

Linear Discriminant Analysis for dimensionality reduction - choosing the dimension

I'm using Linear Discriminant Analysis to do dimensionality reduction of a multi-class data. What is the best method to determine the "correct" number of dimensions? Can I use a method similar to PCA, ...
0
votes
1answer
60 views

Dimensionality reduction: RBM autoencoders vs. de-noising autoencoders

I am looking at non-linear dimensionality reduction techniques and am currently trying to understand the practical differences between different autoencoder approaches: Can somebody point me to a ...
0
votes
1answer
30 views

Help understanding DCT compression for a vector

I have multiple 60-dimensional vectors on which I need to apply DCT and reduce to various dimensions. I'm trying to understand how this happens: ...
2
votes
1answer
147 views

How to use PCA in regression?

I'm currently reading in the Applied Predictive Modeling book about PCA for dimensionality reduction. I've read the following: If the predictive relationship between the predictors and response is ...
0
votes
1answer
37 views

Which is the right way to apply PCA on different sized matrices

I am working on human age classification where I have four descriptors, namely GEI, FED, UC ...
0
votes
0answers
8 views

MFCC and VQ - text independent speaker recognition

Im trying to create models for each person of my system to identify them. I did some preprocessing split sound into frames and extract mfcc. After feature extraction I get matrix ...
2
votes
2answers
69 views

In PCA, what is the connection between explained variance and squared error?

For an observations matrix $X$, I use PCA to reduce the dimension of the data to $L$. I know that in this case, PCA is guaranteed to minimise the mean reconstruction error. In Wikipedia notation, ...
0
votes
1answer
49 views

What to do if number of features is much larger than number of observations?

I have the following concrete example of the problem. Every day I measure 15 different parameters. In the end of the month I have one number (let's call it "target") associated with the given month. I ...
0
votes
0answers
14 views

use of singular value decomposition for dimensionality reduction [duplicate]

I am very new to machine learning. I just understand dimesionality reduction from here I am trying to understand this paper. Here they use Singular value decomposition to reduce dimension. Can ...
2
votes
0answers
20 views

Regulating precision in dimension reduction

I am having a problem trying to minimize the distortion when compressing/decompressing a set of parameters. Let me try to explain. I have a space of parameters $q_1$,$q_2$,...,$q_n$. Since these ...
3
votes
1answer
60 views

Interpret multidimensional scaling plot

I have data with 4 observations and 24 variables. To understand the underlying relationship I performed Multi-Dimensional Scaling (MDS), and got a plot like this: Now the issue is with the correct ...
0
votes
1answer
25 views

Dimension reduction: how do we know whether the reduced variables have already captured most information of the original higher-dimensional variables?

I am working on machine learning methods to do dimensional reduction. And I am wondering are there any ways to determine whether the reduced variables have already captured most information of the ...
0
votes
0answers
10 views

When choosing k biggest eigenvalues for dimension reduction, should I take into account the absolute value?

I want to reduce the dimension of a matrix, I want to take K biggest eigenvalues and vectors that represent 98% of the sum of the eigenvalues. Is this sum taking the absolute value of all ...
0
votes
0answers
22 views

Training for Regression with Multiple Outputs per Input Data

I want to use a neural network (or any other method, for that matter) to perform regression from a high-dimensional space (10k dimensions) to a low-dimensional space (3 dimensions). To train this, I ...
1
vote
1answer
48 views

Dimensional reduction in Self-Organizing Maps: how to map a multidimensional Vector to a low-dimensional Grid

Good Day to everyone. I have spent quite some time now, introducing myself to neural networks. Therefore i am also looking into SOM's. Of course also on this site, as far as i have potentially ...
1
vote
0answers
44 views

Subset selection, shrinkage and dimensionality reduction in regression analysis

I am currently reading An Introduction to Statistical Learning: With Applications in R by Robert Tibshirani and Trevor Hastie. I am confused about various regularization methods for linear regression, ...
1
vote
2answers
36 views

Visualizing Nodes in a Neural Network (Dimensionality Reduction?)

Imagine we are working with the MNIST dataset and creating a neural network with 1 hidden layer. So we have a vector of 784 inputs, 100 hidden nodes, and 10 outputs. If we were to visualize each ...
1
vote
1answer
31 views

Should PC vectors be of unit length, when used for projection?

I am using principal component analysis (PCA) for noise reduction, i.e. projecting a point from the original space to the reduced space, and then back to the original one. For this sort of usage, is ...
3
votes
1answer
274 views

Cannot make this autoencoder network function properly (with convolutional and maxpool layers)

Autoencoder networks seems to be way trickier than normal classifier MLP networks. After several attempts using Lasagne all what I get in the reconstructed output is something that resembles at its ...
2
votes
1answer
68 views

Useful methods to find out variable importance?

I have data with 53 records and 52 variables and want to find a suitable predictive model. I think it makes sense to do some dimension reduction and select only a subset of predictors. My data contain ...
1
vote
0answers
63 views

Is it effective to use one hot encoding of categorical data as input to PCA for anomaly detection?

I have a mixture of numeric and categorical inputs, the categorical inputs are relatively low cardinality (perhaps 10-15). I want to use PCA for anomaly detection, but am not sure how best to encode ...
1
vote
1answer
329 views

How to incorporate PCA step into SVM classification?

I've been using SVM to classify a data set without applying PCA. The classification rate was not bad, but I thought maybe applying PCA increases performance. I have a training set (without labels) ...
2
votes
1answer
60 views

Distance metric invariant to dimensionality?

I'm working on a classification/prediction problem where I have to predict a location of an object. The problem that I have is that for every location, I have a unique and different number of feature ...
0
votes
1answer
43 views

Is weight of evidence and information value a technique of dimension reduction

I am trying to understand the concept of weight of evidence and information value. From what I understand, it is a variable reduction technique where we only use variables with IV > 0.5 in the model. ...
1
vote
0answers
18 views

Dimensionality reduction for narrow, tall matrices

I have a matrix with three columns and a lot of rows. The first two columns contain integers N1 and N2. N1 is always {0,1,2,3}, but N2 can vary from let's say -50 to 50. It's always the same N1s, but ...
1
vote
0answers
54 views

Elastic net regularization - variables penalization

I have a data with ~ 3000 factor predictors with ~ 6 levels, many rows (300k+), and binary Y (trying to predict probability of event). There are many groups of variables that are highly correlated. I ...
2
votes
1answer
92 views

Difference between principal directions and principal component scores in the context of dimensionality reduction

I have performed principal component analysis (PCA) of data matrix $X$ by doing singular value decomposition (SVD) $$ X = U S V', $$ where the columns of $V$ are the principal directions/axes and the ...
0
votes
1answer
43 views

Can I linearly combine the results of PCA with the variance of each feature?

I'm trying to reduce the dimension of a dataset from 8 features to 1 using the principal component analysis (PCA) algorithm. The reduced dataset needs to be in 1 dimension(D) so I can use it for ...
2
votes
0answers
48 views

Best practices for reducing/clustering data which is location/time based?

I have event-based data (or longitudinal data) which has a person's age, gender, location (zipcode etc.), the datetime when they saw something, and how long they watched it. I have a project ...
0
votes
1answer
59 views

Explained variance in dimensionality reduction

I am new to dimensionality reduction and I am trying to learn different techniques about it. I am noticing that, unlike PCA, many other algorithms do not provide the explained variance of each feature ...
0
votes
0answers
60 views

Can Naive Bayes be used with feature hashing and one hot encoding?

I was wondering if a naive bayes implementation can be used with data that has been feature hashes and one hot encoded. The data I am looking at is mobile ad click data from the avazu kaggle ...
3
votes
1answer
166 views

How LDA, a classification technique, also serves as dimensionality reduction technique like PCA

In this article , the author links linear discriminant analysis (LDA) to principal component analysis (PCA). With my limited knowledge, I am not able to follow how LDA can be somewhat similar to PCA. ...
47
votes
10answers
6k views

Explain “Curse of dimensionality” to a child

I heard many times about curse of dimensionality, but somehow I'm still unable to grasp the idea, it's all foggy. Can anyone explain this in the most intuitive way, as you would explain it to a ...