Refers to techniques for reducing a large number of variables to a smaller number while preserving as much information as possible. Prominent methods include PCA, MDS, Isomap, etc.

learn more… | top users | synonyms

0
votes
0answers
11 views

Relation between percentage of variance in first components of PCA and linear separability of a data set?

What we can say about a data set, if we apply PCA on the data set and observe there is high percentage of variance in the first principal component? Can we say this data set is linearly separable? Can ...
1
vote
1answer
23 views

How does one visualize the self-organizing map of $n$-dimensional data

I have a data set consisting from $7$-dimensional data points. I want to produce a self-organizing map for this data to see how my data is clustered. I have been reading some tutorials from the ...
0
votes
1answer
35 views

Why convert categorical data into numerical using one hot encoding

I don't have very strong statistical background, and I'm new in data science... Now, I am practicing PCA (Principle Component Analysis) for dimension reduction. This tutorial looks very complete, but ...
6
votes
1answer
134 views

Is PCA a non-linear transform?

In the article Relative Information Loss in the PCA, the authors make, at some point (in the introductory section), the following statement: In case the orthogonal matrix is not known a priori, ...
0
votes
1answer
38 views

Negative coefficients for ordinal logistic regression in R

I am trying to model my dependent variable (ordinal - three levels) using a set of independent variables (5 ordinal and 10 numeric). I am using lrm function in ...
1
vote
0answers
8 views

Feature extraction from data in the form of many manifolds, in hierarchial structure and various dimensions

Is there a known feature extraction method which was developed to cope with data that satisfies the following assumptions?: The data points are real valued vectors in ...
0
votes
0answers
11 views

How to choose an optimal dimension reduction factor in LSA processing

I'm performing a K-Means clustering on a 400.000 text dataset. After eliminating useless chars and removing stopwords, I get a dictionnary size of around 42000 words. So before doing the clustering, ...
0
votes
0answers
5 views

Filter Feature Selection approaches for continuous variables?

I've noticed that correlation-based filtering for selecting features in high dimensional data require discretization of continuous variables, like e.g. Fast Correlation-based Filtering or regular CFS. ...
5
votes
1answer
131 views

What is the connection between partial least squares, reduced rank regression, and principal component regression?

Are reduced rank regression and principal component regression just special cases of partial least squares? This tutorial (Page 6, "Comparison of Objectives") states that when we do partial least ...
2
votes
1answer
24 views

Self Organizing Maps: How is the location computed and updated?

I have read other similar questions on here, but I am still unsure how SOM deals with the positions/locations of the neurons. Say that the input space is N-dimensional. I initalise some weights, and ...
0
votes
0answers
20 views

PCA + Parallel Analysis

When I realize the Factor Analysis (I have 16 items), the PCA says I have 5 factors. But in the scree plot there is no elbow at all, just a decreasing line, that makes me think maybe I shouldn't be ...
1
vote
0answers
8 views

Understanding the bound given by Johnson–Lindenstrauss lemma

This was a question I asked in the mathematics site. Though no one replied but I have figured it out. But new questions arise. Former Question: Here I choose to use the statement made by S.Dasgupta: ...
0
votes
0answers
16 views

Help in understanding hashing for nearest neighbor search

Hashing is a technique for large- scale visual search and a variety of hashing-based method- s have been proposed Survey paper : Hashing for similarity search . The application of hashing to ...
4
votes
1answer
322 views

Combining multiple variables into one “score”

My question is very similar to this one, which was not solved unfortunately. I am working on a project for which I want to rank countries by means of their HIV/AIDS burden. So I collected a lot of ...
1
vote
0answers
11 views

Label reduction on dataset

This question related to this other one, for which I have devised a strategy and now want some feedback on it. My data consists of 434042 rows, each corresponding to an observation tagged with 1 of ...
4
votes
0answers
36 views

Why would I ever use a linear autoencoder for dimensionality reduction?

Following on from: What're the differences between PCA and autoencoder? If I want to do dimensionality reduction and restrict myself to using a linear activation for my autoencoder, is there any ...
0
votes
0answers
30 views

Reducing number of labels in a dataset

I have a dataset that contains of 13 variables, of which: 5 are binary (T/F). 4 are categorical. 1 is ordinal (ranking). 1 is continuous. 1 is the time component. 1 is a final categorical variable, ...
0
votes
0answers
36 views

Is there a supervised/semi supervised version of pca for dimensionality reduction?

PCA can give me the proper result if "Large variances have important dynamics" holds true for the data. In other words if I want to know along which components the variance of my data is maximized, ...
4
votes
3answers
374 views

random forest - summarize two features in one without losing information

I am training a random forest on a dataset including both categorical and numerical features. In particular I have a binary feature, call it $x_1$, which has $0$ or $1$ as possible outcomes. I also ...
0
votes
0answers
13 views

Understanding filter space in convolutional neural networks and its reduction in Inception architecture

From this source I acquired a quite good understanding of 1x1 convolutions in Inception CNN and how they perform a reduction in the filters dimension. There is one thing I would like to clarify ...
3
votes
0answers
19 views

Data reduction and xgboost(or other boosting and decisision tree methods)

I wonder, does data reduction(ex:factor analysis) have an impact on the result of boosting(ex:xgboost) or decision trees methods other than time gain?
1
vote
0answers
33 views

How much data versus dimensions are needed to overcome the curse of dimensionality?

Are there any guidelines for knowing how many training samples are needed based on the number of features you have in order to not have accuracy degradation as a result of the curse of dimensionality? ...
0
votes
0answers
15 views

Correlation between matrix variables and its PCA scores

Suppose a matrix $X$ and $T$ the score matrix obtained from a PCA decomposition of $X$. Denote as $x_i$ and $t_i$ the columns of $X$ and $T$ respectively. Is there any reason for which $cor(x_i, ...
1
vote
1answer
32 views

How to reduce dimensionality of audio data that comes in form of matrices and vectors?

I'm working on a project involved with identifying different types of sounds (such as screams, singing, and bangs) from each other. We've got our data a reasonable number of different transformations ...
0
votes
0answers
26 views

Kernel PCA increases dimensionality compared with PCA?

I was trying to use sklearn to perform kernel PCA with 28*28 = 784 dims data. At first I used PCA to reduce dimensionality and I chose to reduce to k dimensions where k could explain 95% of the ...
1
vote
0answers
25 views

Handling datasets with categorical variables of many levels

I am working on the Diabetes in 130 US hospitals for years 1999--2008 dataset. After removing unnecessary variables (i.e. some IDs or near-zero-variance variables) and doing some naive imptuation, I ...
0
votes
0answers
31 views

Text Mining Using irlba for dimension reduction

I am trying to do some dimension reduction on a sparse matrix I have. The data is text data currently formatted in a document term matrix. I did some reading and used the irlba package to reduce the ...
3
votes
1answer
54 views

Does it make sense to use PCA when the determinant of the correlation matrix is (almost) zero?

I'm running a PCA over a data set of $N \times p$ size ($N\approx 1000$ being the number of measurements and $p\approx 200$ being the number of dimensions/predictors). I expect many of the predictors ...
0
votes
0answers
26 views

Can someone please explain about squared loading values in ClustOfVar package

I am using ClustOfVar package in R to cluster both nominal and interval variables for dimensionality reduction. Can someone please explain what is squared loading values. how to pick representative ...
1
vote
2answers
67 views

Projecting to lower/higher-dimensional space for classification: dimensionality reduction vs kernel trick

Whilst learning about classification, I have seen two different arguments. One is that projecting the data to a lower-dimensional space, such as with PCA, makes the data more easily separable. The ...
0
votes
0answers
7 views

how to define which of measured, non gaussian variables are effective in discriminate given groups?

I have eight groups (100 samples each) and 43 evaluated variables, with different distributions (some of them are right skewed, some left skewed, some with many zeros etc.). I'm trying to understand ...
0
votes
0answers
75 views

A technique like truncated SVD that uses non-orthogonal components?

Question: Is there a technique like truncated SVD that instead of orthogonal components relies on non-orthogonal components? Specifically, just like a truncated SVD computes simultaneously the ...
2
votes
1answer
95 views

Is each of the PCA or PLS components just one of the original variables?

I am confused about what a component is in PCA and PLS. Are the components just the original variables but not necessarily in the same order? For example, in PCA, if I had 8 variables in my data, ...
1
vote
0answers
40 views

Ranking of neurons of bottleneck layer in AutoEncoder network, similar to components in PCA

When using Principal Component Analysis (PCA) for dimension reduction, the extracted components (basis vectors) are sorted based on the eigenvalues of the covariance matrix of data, i.e. we have the ...
0
votes
0answers
23 views

How exactly is dependent variable expressed in terms of independent variables using Partial Least Square Regression? [duplicate]

I understand the working of NIPALS algorithm but while doing the regression using PLS how exactly the relation between known and unknown is established using Principle Component Analysis. The idea is ...
0
votes
1answer
56 views

LDA for dimensionality reduction usage

I have a original dataset with 70 samples, each sample with 96 features. The samples are labeled as 0 or 1. So I use linear discriminant analysis (LDA) to reduce the dimensionality of all the dataset, ...
0
votes
0answers
14 views

sparse principal component analysis

I have a dataset includes of evaluated KPIs collected from experts. It has 67 dimensions or variables(KPIs) and the number of evaluated data that I have collected is around 50 for each KPIs. I want to ...
1
vote
1answer
19 views

How to remove the effect of other dimensions?

I have data of 3 Dimensions. the value of the third dimension is highly correlated with the value of the first two dimensions. I want to remove the effect of the first two dimensions on the third ...
0
votes
1answer
48 views

Performance of a classifier change heavily

I'm using a data set of 32 face persons and a svm-rbf to classify and a random group of 70% for train and 30% for test. The problem is that my results are heavily dependent of the random group used ...
0
votes
1answer
51 views

Dimensionality reduction for high dimensional sparse data before clustering or spherical k-means?

I am trying to build my first recommender system where i create a user feature space and then cluster them into different groups. Then for the recommendation to work for a particular user , first i ...
3
votes
0answers
26 views

Rules of thumb for a proportion of outliers depending on the dimension

I am implementing and benchmarking different "robust" PCA (principal component analysis, see for instance Robust Principal Component Analysis?) for data that should align (I have no prior on the ...
3
votes
0answers
57 views

How to do dimensionality reduction on a huge data set?

I am working with fMRI data of ~1000 subject. Each subject has a feature vector of ~150 million dimension. So I can only keep the feature vectors of ~10 subjects in memory. What are some algorithms ...
2
votes
1answer
95 views

Meaning of “reconstruction error” in PCA and LDA

I am implementing PCA, LDA, and Naive Bayes, for compression and classification respectively (implementing both an LDA for compression and classification). I have the code written and everything ...
1
vote
1answer
91 views

Can PCA be applied twice or more? [duplicate]

I have a very high-dimensional dataset with 27k features. I want to a reduce the dimensionality of the dataset. I want to use PCA to reduce the dimensionality to 2 as the toolbox I am using expects 2 ...
0
votes
0answers
31 views

Units of measurement on a two-dimensional PCA scatterplot

I have a dataset that consists of the average consumption of 200~ types of food in grams per person per week for several countries, much like the dataset described at the bottom of this page Eating in ...
0
votes
1answer
66 views

data normalization after dimension reduction for classification

The classifier is KNN or RBF-SVM. After doing dimension reduction (e.g., PCA, LDA or KPCA, KLDA), does it need to do normalization before classification? In LIBSVM ...
0
votes
0answers
33 views

Inferences from PCA plot

I have done a dimensionality reduction of binary labelled data (0,1 labels) from 300 features to 2 features. The plot looks like - What kind of inferences can I make from this plot? Can I infer - ...
0
votes
0answers
49 views

How to compare PCA with KPCA for dimension reduction?

Both linear principal component analysis (PCA) and kernel principal component analysis (KPCA) are unsupervised dimension reduction methods. I have a dataset with $4000$ training samples and $40000$ ...
0
votes
0answers
19 views

t-SNE: Aren't identical words meant to stack in the same place?

I looked at t-SNE examples on Laurens van der Maatens' page and noticed, that some words/phrases appear multiple times in slightly different locations (for example zoom in at t-SNE example, middle to ...
0
votes
1answer
137 views

How does Principal Component Analysis help me understand my data?

I have a dataset which contains 10000 examples. Each example has 100 dimensions. These dimensions have the same scale. I clustered all examples using their 100-dimensional vectors and drew the elbow ...