Dimensionality reduction refers to techniques for reducing many variables into a smaller number while keeping as much information as possible. One prominent method is [tag pca]

learn more… | top users | synonyms

23
votes
8answers
3k views

Explain “Curse of dimensionality” to a child

I heard many times about curse of dimensionality, but somehow I'm still unable to grasp the idea, it's all foggy. Can anyone explain this in the most intuitive way, as you would explain it to a ...
0
votes
1answer
32 views

Fisher Projection vs Linear Discriminant Analysis [on hold]

Basically, I am confused between Fisher and LDA. Looking for differences between the two. How is the Fischer projection computed in R?
0
votes
0answers
16 views

How to analyse the influence of 100 categorical or continuous predictors on one continuous response?

I am analysing a genetic dataset that consists of 288 individuals, 100 genetic markers as predictors and one continuos variable (day of death) as outcome. Each predictor has three categories or ...
0
votes
1answer
12 views

Inverse tSNE is feasible?

Short question: is it meaningful to use tSNE ( http://homepage.tudelft.nl/19j49/t-SNE.html) to modify existing high-dimensional data using similarities in some low-dimensional vectors? In essence, ...
0
votes
0answers
11 views

What is the best way to apply a dimension reduction to a time series, and not to be affected by the outliers?

I want to apply a dimension reduction to a time series, in order to not have a high dimensional one, but I don't want this transformation to be affected by the outliers my time series could have.
0
votes
0answers
17 views

Fusion of two model reduced sensing systems / Comparison of two models

General context: I have a computer vision problem where I take an image sample then: Analyse* it (details omitted) to obtain a vector of values Run PCA (Principle Component Analysis) on it to ...
1
vote
1answer
42 views

PCA to reduce dimensionality then classifier of choice?

I read about PCA online and the way it computes a covariance matrix, computes eigenvalues, and then transforms the matrix to reduce the dimensionality of the matrix to a certain number k <= p. ...
0
votes
0answers
20 views

What is correct implementation of LDA (Linear Discriminant Analysis)? [migrated]

I found that the result of LDA in OpenCV is different from other libraries. For example, the input data was ...
1
vote
0answers
41 views

Normalisation before and after PCA

Is it valid to normalise a dataset, reduce dimensionality with PCA and then to normalise the reduced dimension data? Assuming this is performed on training data, should the same PCA coefficients be ...
0
votes
0answers
17 views

Should data be centered+scaled before applying t-SNE?

Some of my data's features have large values, while other features have much smaller values. Is it necessary to center+scale data before applying t-SNE to prevent bias towards the larger values? I ...
0
votes
0answers
24 views

LDA scores too big

I'm trying to do dimensionality reduction with linear discriminant analysis (LDA) in MATLAB. I'm using this code to calculate the coefficients. But I'm confused whether (and when) should I center the ...
1
vote
1answer
27 views

How to apply distance-based clustering or dimensionality reduction for too many samples

I have a dataset with 200K samples (cases) and 30 variables. Every distance-based method for clustering or dimension reduction technique that I use, such as DBSCAN, Hierarchical Clustering, LLE, ...
0
votes
0answers
24 views

Data sets for which PCA can classify better than LDA (using a very small training set)

Can you provide an example of a dataset where PCA can find better discriminant directions than (LDA) Linear Discriminant Analysis? One example is UCI's wine data set. If you use only 2 observations ...
5
votes
1answer
180 views

Is large scale PCA even possible?

The principal component analysis (PCA) algorithm assumes that columns of an input matrix have zero mean. This can be achieved easily, but when the input matrix is sparse, the centered matrix will now ...
1
vote
0answers
31 views

Embedding in machine learning

What does word 'embedding' mean in machine learning? As I understand it is finding intrinsic dimensionality of the data. But how it works practically? Specifically, how does Gradient Boosted ...
1
vote
1answer
55 views

Evaluate output of different dimensionality reduction methods

I used PCA, ICA, and FA to perform dimensionality reduction on my data. How can I measure which method performed best? If I reduce my data to 3 dimensions and plot it, what type of trends would ...
0
votes
0answers
45 views

How to quantify performance of Linear Discriminant Analysis (LDA)?

I have implemented Linear Discriminant Analysis (LDA) for dimensionality reduction in C. But I don't know how to quantify performance of the LDA. Could someone help me?
1
vote
0answers
23 views

Techniques for plotting PCA projections in more than three dimensions

After running PCA on my data set, I noticed that using the three first eigenvectors, a separation between two different classes is still achievable (doing PCA on data from two classes). Unfortunately, ...
1
vote
0answers
23 views

Uniformly sampling principal component scores to explore response surface

New simpler version of the question Consider a sample $\mathbf{X} \in \mathbb{R}^{n\times p}$ of $n$ points in $\mathbb{R}^p$ with $p$ small, say $p=5$, and $n$ large, say $n=3000$. Because they are ...
3
votes
1answer
72 views

Supervised dimensionality reduction

I have a data set consisting of 15K labeled samples (of 10 groups). I want to apply dimensionality reduction into 2 dimensions, that would take into consideration the knowledge of the labels. When I ...
4
votes
3answers
182 views

Would PCA work for boolean (binary) data types?

I want to reduce the dimensionality of higher order systems and capture most of the covariance on a preferably 2 dimensional or 1 dimensional field. I understand this can be done via principal ...
1
vote
0answers
31 views

How to deal with different sizes of sentences when giving them as input to a Neural Network?

I am giving a sentence as input to a tree structured Neural Network, where the leaf nodes will be the word vectors of the words in the sentence. That tree will be a binarized constituency(see the ...
0
votes
0answers
27 views

For a low-rank regularized PCA, what is the limit of dimension reduction for a given p and n of data?

Here p is the dimension of data, and n is the number of data rows, so the data matrix is a $n∗p$, and if we use PCA for dimension reduction, and in this case it is a low-rank regularized PCA, what ...
2
votes
1answer
37 views

“Balancing” principal components

I apologize in advance for the poorness of my statistics and mathematics. I am doing PCA on data (emission spectra) that I know a priori should have two strong components (there are two fluorescent ...
0
votes
1answer
22 views

Dimensionality Reduction on a single Character Vector

I have a dataset I'm using to predict a binary outcome variable with 6 columns. Five of them are 10-30 level categorical variables with information about the user, e.g. job function, industry, ...
0
votes
0answers
2 views

Looking for methods to compare two knowledge bases

I'm working on a model for communication between two computer agents to generate collaborative narratives. These agents have different knowledge bases (KB) and I'm interested on determining how ...
1
vote
0answers
27 views

How does t-SNE slow down with increasing number of dimensions?

I'm trying to understand the computational bounds of t-SNE. It's learned with SGD, so it'll have to go through some number of gradient-descent iterations. We can ignore that here, and focus on the ...
0
votes
0answers
57 views

Sparse features and dimension reduction

Let sparse feature be a feature which values are subsets of some set. For example, the set of countries from which user logged to server is a sparse feature, because for each user we've got the set of ...
0
votes
0answers
28 views

Nonlinear dimensionality reduction (sample size is smaller than number of features)

One question for the nonlinear dimensionality reduction. I have 800 samples and 4900 features for a regression problem, 80% for training and 20% for testing. I have tried linear PCA to reduce it to ...
2
votes
1answer
79 views

What are the differences between autoencoder and t-SNE?

As far as I know, both autoencoder and t-SNE are used for nonlinear dimension reduction. What are the differences between them and why should I use one versus another? thanks!
4
votes
1answer
45 views

Does rank of observation matrix tell anything useful when applying machine learning?

Suppose I have an observation matrix of size $N \times M$ where $N$ is the number of samples and $M$ is the number of variables. If the rank of the observation matrix is $R<M$, does it tell ...
7
votes
1answer
318 views

What is the intuitive reason behind doing rotations in Factor Analysis/PCA & how to select appropriate rotation?

My Questions What is the intuitive reason behind doing rotations of factors in factor analysis (or components in PCA)? My understanding is, if variables are almost equally loaded in the top ...
0
votes
2answers
65 views

Using k-means for reducing the size of the training set of a Kernel SVM

I have a classification problem with the following characteristics: a few million data points around one hundred features non-linearly separable Training a SVM with an RBF Kernel is not feasible ...
0
votes
1answer
77 views

Doubt with a distance based Redundancy analysis

I conducted a distance based redundancy analysis (dbRDA) to explore the relevance of some environmental variables in explaining the patterns of the distribution (i.e., spatial and temporal) of two ...
3
votes
1answer
182 views

Why is the curse of dimensionality also called the empty space phenomenon?

The curse of dimensionality refers to the fact that the huge number of correlated features tends to increase the complexity of the treatment that has to be applied to the data set. This is also called ...
5
votes
1answer
111 views

How to add a third variable to a bar plot?

I'm trying to find the best way to show the following data: ...
1
vote
0answers
17 views

Practical applications of dimensionality reduction methods: Filtering, Wrapper, Embedded models

Filtering is basically sorting some features and picking top performing ones. Where wrapper in wrapper we go through unused/used features add/remove some of them and test performance over validation. ...
2
votes
1answer
52 views

How to measure loss of performance of clustering by applying dimensionality reduction

Let's suppose I have a given dataset with $n$ features. Having a data-centric approach, I would like to measure the loss of performance of applying a given dimensionnality reduction technique, for a ...
0
votes
0answers
10 views

Isomap and Local MDS- embedding to linear spaces or not?

In Section 14.9 [1], it is said that Isomap and LocalMDS are embeddings into non-linear manifolds. The embedding is clearly a non linear operation, thus worthy of being NLDR. But As both Isomap and ...
3
votes
1answer
80 views

Similarities and dissimilarities in classical multidimensional scaling

I am having trouble reconciling between several terms in MDS. According to [1], Section 14.8, Classical MDS takes similarities as inputs. In [2], also cited in Wikipedia, Classical MDS takes ...
1
vote
0answers
57 views

Reducing number of variables for Independent Component Analysis

Say I have a dataset with $n$ observations of $p$ random variables. Since $p$ is "large" (mine is 72), I would like to perform a fastICA only on a subset of $k$ variables, maintaining the same number ...
0
votes
1answer
51 views

Naive Bayes Binary Classification with Binary Features

I have a dataset with two classes $C_0$ and $C_1$. I have around $10$ to $20$ features that take binary values (either $0$ or $1$). My dataset has around $10000$ instances, with only a hundred of ...
2
votes
0answers
79 views

Structure of semantic relationships using Latent Semantic Analysis [closed]

I am struggling to answer the below question: How would you describe the structure of semantic relationships among the terms from a document collection using principles of Latest Semantic Analysis? ...
1
vote
0answers
45 views

Principled way of collapsing categorical variables with many categories

What techniques could I use to optimize the collapsing of many categories to a few, for the purpose of using them as an input to a statistical model? Consider a variable like college student major. ...
0
votes
0answers
246 views

What's wrong with my derivation on stochastic neighbor embedding?

I've been reading the well-known stochastic neighbor embedding [https://www.cs.nyu.edu/~roweis/papers/sne_final.pdf] for a long time. And I've been trying to derive the gradient of its optimization ...
0
votes
1answer
57 views

Classifier with variable number of features

I am trying to make a classifier when each sample has a variable number of features. An example of how this could occur is, for example, if the features are the purchases (type, dollar amount, etc) ...
0
votes
1answer
61 views

PCA/MFA for (graphical) dimension reduction: what to do with very small explained variance?

I ran a Multiple Factor Analysis on a data set with 3,924 rows and 96 columns, of which six are (unordered) categorical, with 12-14 categories in each, and the rest are numeric, mean-centered and ...
0
votes
3answers
74 views

How To Determine The Number Of Dimensions To A Machine Learning Problem

I have a bit to learn about machine learning, so please pardon me if I am asking the wrong type of question. I have read some about neural networks and SVMs, so I'm not completely in the dark. I am ...
3
votes
1answer
92 views

Interpreting standard deviation for PCA

I'm running PCA on my dataset using r and need some help interpreting the standard deviation results. Here are the results ...
0
votes
0answers
39 views

How to interpret PCA plots made using R [duplicate]

I'm using PCA for the first time and just experimenting with it. I used PCA on my dataset that can be found here ...