Dimensionality reduction refers to techniques for reducing many variables into a smaller number while keeping as much information as possible. One prominent method is [tag pca]

learn more… | top users | synonyms

0
votes
0answers
17 views

What are the differences between autoencoder and t-SNE?

As far as I know, both autoencoder and t-SNE are used for nonlinear dimension reduction. What are the differences between them and why should I use one versus another? thanks!
4
votes
1answer
36 views

Does rank of observation matrix tell anything useful when applying machine learning?

Suppose I have an observation matrix of size $N \times M$ where $N$ is the number of samples and $M$ is the number of variables. If the rank of the observation matrix is $R<M$, does it tell ...
6
votes
1answer
148 views

What is the intuitive reason behind doing rotations in Factor Analysis/PCA & how to select appropriate rotation?

My Questions What is the intuitive reason behind doing rotations of factors in factor analysis (or components in PCA)? My understanding is, if variables are almost equally loaded in the top ...
0
votes
2answers
40 views

Using k-means for reducing the size of the training set of a Kernel SVM

I have a classification problem with the following characteristics: a few million data points around one hundred features non-linearly separable Training a SVM with an RBF Kernel is not feasible ...
0
votes
1answer
31 views

Doubt with a distance based Redundancy analysis

I conducted a distance based redundancy analysis (dbRDA) to explore the relevance of some environmental variables in explaining the patterns of the distribution (i.e., spatial and temporal) of two ...
2
votes
1answer
93 views

Why is the curse of dimensionality also called the empty space phenomenon?

The curse of dimensionality refers to the fact that the huge number of correlated features tends to increase the complexity of the treatment that has to be applied to the data set. This is also called ...
5
votes
1answer
81 views

How to add a third variable to a bar plot?

I'm trying to find the best way to show the following data: ...
1
vote
0answers
9 views

Practical applications of dimensionality reduction methods: Filtering, Wrapper, Embedded models

Filtering is basically sorting some features and picking top performing ones. Where wrapper in wrapper we go through unused/used features add/remove some of them and test performance over validation. ...
2
votes
1answer
32 views

How to measure loss of performance of clustering by applying dimensionnality reduction

Let's suppose I have a given dataset whit $n$ features. Having a data-centric approach, I would like to measure the loss of performance of applying a given dimensionnality reduction technique, for a ...
0
votes
0answers
8 views

Isomap and Local MDS- embedding to linear spaces or not?

In Section 14.9 [1], it is said that Isomap and LocalMDS are embeddings into non-linear manifolds. The embedding is clearly a non linear operation, thus worthy of being NLDR. But As both Isomap and ...
3
votes
1answer
57 views

Similarities and dissimilarities in classical multidimensional scaling

I am having trouble reconciling between several terms in MDS. According to [1], Section 14.8, Classical MDS takes similarities as inputs. In [2], also cited in Wikipedia, Classical MDS takes ...
1
vote
0answers
54 views

Reducing number of variables for Independent Component Analysis

Say I have a dataset with $n$ observations of $p$ random variables. Since $p$ is "large" (mine is 72), I would like to perform a fastICA only on a subset of $k$ variables, maintaining the same number ...
0
votes
1answer
29 views

Naive Bayes Binary Classification with Binary Features

I have a dataset with two classes $C_0$ and $C_1$. I have around $10$ to $20$ features that take binary values (either $0$ or $1$). My dataset has around $10000$ instances, with only a hundred of ...
2
votes
0answers
68 views

Structure of semantic relationships using Latent Semantic Analysis [on hold]

I am struggling to answer the below question: How would you describe the structure of semantic relationships among the terms from a document collection using principles of Latest Semantic Analysis? ...
1
vote
0answers
12 views

Principled way of collapsing categorical variables with many categories

What techniques could I use to optimize the collapsing of many categories to a few, for the purpose of using them as an input to a statistical model? Consider a variable like college student major. ...
0
votes
0answers
229 views

What's wrong with my derivation on stochastic neighbor embedding?

I've been reading the well-known stochastic neighbor embedding [https://www.cs.nyu.edu/~roweis/papers/sne_final.pdf] for a long time. And I've been trying to derive the gradient of its optimization ...
0
votes
1answer
32 views

Classifier with variable number of features

I am trying to make a classifier when each sample has a variable number of features. An example of how this could occur is, for example, if the features are the purchases (type, dollar amount, etc) ...
0
votes
1answer
40 views

Justification for variable reduction by removing predictors with near zero variance

I have a large number of variables that I'm trying to reduce, and I've stumbled on Kuhn's (2008) suggestion that I eliminate variables with zero or near-zero variance. This makes sense to me, it's ...
0
votes
1answer
40 views

PCA/MFA for (graphical) dimension reduction: what to do with very small explained variance?

I ran a Multiple Factor Analysis on a data set with 3,924 rows and 96 columns, of which six are (unordered) categorical, with 12-14 categories in each, and the rest are numeric, mean-centered and ...
0
votes
3answers
59 views

How To Determine The Number Of Dimensions To A Machine Learning Problem

I have a bit to learn about machine learning, so please pardon me if I am asking the wrong type of question. I have read some about neural networks and SVMs, so I'm not completely in the dark. I am ...
3
votes
1answer
48 views

Interpreting standard deviation for PCA

I'm running PCA on my dataset using r and need some help interpreting the standard deviation results. Here are the results ...
0
votes
0answers
38 views

How to interpret PCA plots made using R [duplicate]

I'm using PCA for the first time and just experimenting with it. I used PCA on my dataset that can be found here ...
0
votes
0answers
38 views

Is there a known relationship between the Intrinsic Dimensionality of a dataset and the VC dimension of a model?

We know that the Intrinsic Dimension of a dataset gives the low dimensional sub manifold in which the real data distribution lies. On the other hand, the VC dimension of a model gives the bounds for ...
0
votes
0answers
19 views

Why is feature normalization important in PCA? [duplicate]

If feature normalization is not performed, does the algorithm give incorrect results or is it it inefficient or both?
0
votes
0answers
17 views

Forward sequential feature selection improving classifier performance?

I was in a bit of a conversation with a co-worker about using forward selection. My training data is on order of ~6,000 w/ dimensionality of 1,200, and testing data on order of ~3,000. Currently, I'm ...
2
votes
1answer
62 views

Manifold learning: does an embedding function need to be well behaving?

I am trying to learn about manifold learning techniques; a family of methods in machine learning. According to this idea, there is a low ($d$) dimensional, hidden space where the real data generation ...
3
votes
1answer
91 views

Two broad categories of dimensionality reduction

As a starter in dimensionality reduction (DR), I recently acquired the following understanding. There are two very broad categories of DR techniques: We can compute an analytic form of mapping from ...
0
votes
0answers
23 views

What is the heuristic to decide number of components for LDA dimensionality reduction?

In the PCA case, I prefer to plot the variance and choose number of components regarding that plot's breaking point. In the LDA (linear disriminant analysis) case, what can be used for such an ...
0
votes
0answers
25 views

How to determine which variables need to be trimmed in PCA or Factor analysis?

Background: I'm working with log returns for about 400 tech stocks. I want to use PCA to reduce these into principal components (Internet companies, software developers, circuit board manufacturers, ...
8
votes
4answers
193 views

How can top $k$ principal components retain the predictive power on a dependent variable?

Suppose I am running a regression $Y \sim X$. Why by selecting top $k$ principle components of $X$, does the model retain its predictive power on $Y$? I understand that from ...
1
vote
0answers
24 views

Standardizing dimension reduction output

I understand that data is (typically) standardized (i.e. zero mean and unit variance) before dimension reduction technique such as PCA/LDA is applied. In addition to this, would it ever make sense to ...
1
vote
0answers
27 views

What is best practise for dimensionality reduction in rows of data

I was wondering what was best practise for dimensionality reduction in observations (as opposed to features) in a data-set? I often have data comprising of a multiple, random number of observations ...
2
votes
1answer
54 views

Can I apply factor analysis on multiple choice questions?

I am looking to validate a questionnaire and would like to know if I can use factor analysis on the multiple-choice questions (MCQ). Also, I have another section where I am asking about participants ...
3
votes
1answer
99 views

When should I use feature selection and when should I use dimensionality reduction techniques?

When should I use feature selection and dimensionality reduction? I know that feature selection is different from dimensionality reduction. But I don't know under what circumstances should I use ...
0
votes
1answer
75 views

Model Selection and RFE using caret

I'm faced with a high dimensional (samples=148, features=20000), supervised binary classification problem. Which I would like to approach with an ensemble of classifiers, that will classify using a ...
0
votes
0answers
42 views

In PCA, can the values in the principle component vectors which are close to zero be removed to see the important features? [duplicate]

In PCA, when I extract the principle component vectors, I am choosing the first vector with the largest corresponding eigenvalue. I notice that some of the values in this vector are close to zero. Can ...
1
vote
0answers
46 views

Dimension reduction for discrete qualitative and aggregated variables

I know about PCA for multiple dimensions of continuous features but here is a problem I have some trouble to find a method for. I don't have a list of individual countries but rather a discrete ...
2
votes
1answer
67 views

Support Vector Machines and the curse of dimensionality

I am reading this paper: "Automated MR image classification in temporal lobe epilepsy", by Focke et al. NeuroImage, 2012. The authors use support vector machines to classify subjects between healthy ...
0
votes
0answers
39 views

Are there any versions of t-SNE for streaming data?

My understanding of t-SNE and the Barnes-Hut approximation is that all data points are required so that all force interactions can be calculated at the same time and each point can be adjusted in the ...
0
votes
0answers
29 views

In R, is there an exact method for the political compass test?

In R, I am looking, in an exact way, for the method (or packages) used on the political compass test (see politicalcompass.org in the link Take the test) In my case, I have one data set consisting in ...
3
votes
1answer
83 views

What is the difference between feature selection and dimensionality reduction?

I know that both feature selection and dimensionality reduction aim towards reducing the number of features in the original set of features. What is the exact difference between the two if we are ...
0
votes
0answers
25 views

PCA reduction and low-reliability components

I'm working on a survey with 288 observation in total (108 complete answers used) and around 200 variables. I'm working on reducing those number using Principal Components Analysis, using R. Suppose ...
1
vote
0answers
67 views

How to reduce dimension of the sampling procedure?

I am stuck with this problem for a long time, hopefully I can get help here! Basically, I want to sample from a posterior distribution that looks like, \begin{align*} X &\sim ...
0
votes
1answer
58 views

LDA vs. SVM for Dimensionality Reduction

Whats the difference between LDA and Linear SVM for dimensionality reduction. I am little confuse as LDA also looks for projection that separates the classes of data and SVM we also look for ...
1
vote
0answers
40 views

Feature space reduction for tag prediction

[x-post] from stackoverflow. I am writing a ML module (python) to predict tags for a stackoverflow question (tag + body). My corpus is of around 5 million questions with title, body and tags for ...
0
votes
1answer
52 views

technical issues regarding to cluster analysis

Hi I would like to seek help with my cluster analysis using SAS. The main objective of the task is to segment customers into groups based on their similarity. The dataset contain mixed types of ...
1
vote
1answer
58 views

Intrinsic dimensionality estimation using Laplacian Eigenmaps

I learnt that I can look at the eigenspectrum of the kernel matrices computed by nonlinear spectral techniques in order to estimate the intrinsic dimensionality of a data-set. I use drtoolbox (The ...
0
votes
1answer
49 views

pruning Neural Network

Since a feedforward NN with a logistic function as activation function is not linear, does it make sense to reduce variables first with principal components or discriminant analysis? Because ...
2
votes
1answer
35 views

Clustering Consumer data with over 100 variables and 50000 rows each

I am tasked with performing a clustering exercise for a consumer survey dataset with the hopes of finding distinct consumer segments. In the past, I've done it using a variety of techniques- ...
0
votes
0answers
15 views

Dealing with seasonality when doing dimensionality reduction

I want to perform dimensionality reduction (in particular, PCA) on a data set that is highly seasonal. One approach that I came across when researching this is "seasonal PCA", where you split your ...