Refers to techniques for reducing a large number of variables to a smaller number while preserving as much information as possible. Prominent methods include PCA, MDS, Isomap, etc.

learn more… | top users | synonyms

0
votes
0answers
20 views

2 features and 2 principal components

Whats the difference between 2 features and 2 principal components? I know what a PCA is, I just have the following problem: If my data has 2 features, the PCA will produce 2 components. So why does ...
0
votes
1answer
28 views

Feature selection: PCA vs intuition? [on hold]

Which one should I choose? How can I combine them (i.e. in series or parallel)? What if there are dummy features in my data? What if my intuition messes things up?
-1
votes
1answer
35 views

When can clustering be used for dimensionality reduction? [closed]

Can a clustering method be used for dimensionality reduction? I though the answer would be that the cluster numbers can act as the synthetic reduced dimension -- but the other day a friend had a more ...
0
votes
1answer
126 views

PCA vs FA vs ICA for dimensionality reduction in questionaire data

I am trying to identify personality traits underlying the multidimensional data from a questionnaire. In more abstract terms this means reducing the dimensionality of my data from N-dimensional (where ...
3
votes
1answer
64 views

Do I need to run PCA over all predictors in a regression model? Can I run it only over the continuous ones?

I'm looking at the Lending Club data from Kaggle and I'm just building a pretty simple model to predict defaults. The data has a large amount of both continuous and categorical variables (I have ...
1
vote
0answers
31 views

How to determine time complexity of EM algorithm of probabilistic PCA?

I was studying probabilistic PCA from Bishop's book. There an EM algorithm is provided to calculate principal subspace: Here $\mathbf M$ is $M\times M$ matrix, $\mathbf W$ is $D\times M$ matrix ...
2
votes
0answers
55 views

If almost all variance is explained by the few first principal components, what can we say about the dataset?

What can we say about a dataset if we apply PCA and observe that there is a high percentage of variance in the first principal component(s)? Can we say that this dataset has linear structure? Can we ...
1
vote
1answer
29 views

How does one visualize the self-organizing map of $n$-dimensional data

I have a data set consisting from $7$-dimensional data points. I want to produce a self-organizing map for this data to see how my data is clustered. I have been reading some tutorials from the ...
0
votes
1answer
57 views

Why convert categorical data into numerical using one hot encoding

I don't have very strong statistical background, and I'm new in data science... Now, I am practicing PCA (Principle Component Analysis) for dimension reduction. This tutorial looks very complete, but ...
6
votes
1answer
150 views

Is PCA a non-linear transform?

In the article Relative Information Loss in the PCA, the authors make, at some point (in the introductory section), the following statement: In case the orthogonal matrix is not known a priori, ...
0
votes
1answer
41 views

Negative coefficients for ordinal logistic regression in R

I am trying to model my dependent variable (ordinal - three levels) using a set of independent variables (5 ordinal and 10 numeric). I am using lrm function in ...
1
vote
0answers
8 views

Feature extraction from data in the form of many manifolds, in hierarchial structure and various dimensions

Is there a known feature extraction method which was developed to cope with data that satisfies the following assumptions?: The data points are real valued vectors in ...
0
votes
0answers
11 views

How to choose an optimal dimension reduction factor in LSA processing

I'm performing a K-Means clustering on a 400.000 text dataset. After eliminating useless chars and removing stopwords, I get a dictionnary size of around 42000 words. So before doing the clustering, ...
0
votes
0answers
7 views

Filter Feature Selection approaches for continuous variables?

I've noticed that correlation-based filtering for selecting features in high dimensional data require discretization of continuous variables, like e.g. Fast Correlation-based Filtering or regular CFS. ...
6
votes
1answer
148 views

What is the connection between partial least squares, reduced rank regression, and principal component regression?

Are reduced rank regression and principal component regression just special cases of partial least squares? This tutorial (Page 6, "Comparison of Objectives") states that when we do partial least ...
2
votes
1answer
26 views

Self Organizing Maps: How is the location computed and updated?

I have read other similar questions on here, but I am still unsure how SOM deals with the positions/locations of the neurons. Say that the input space is N-dimensional. I initalise some weights, and ...
0
votes
0answers
21 views

PCA + Parallel Analysis

When I realize the Factor Analysis (I have 16 items), the PCA says I have 5 factors. But in the scree plot there is no elbow at all, just a decreasing line, that makes me think maybe I shouldn't be ...
1
vote
0answers
9 views

Understanding the bound given by Johnson–Lindenstrauss lemma

This was a question I asked in the mathematics site. Though no one replied but I have figured it out. But new questions arise. Former Question: Here I choose to use the statement made by S.Dasgupta: ...
0
votes
0answers
17 views

Help in understanding hashing for nearest neighbor search

Hashing is a technique for large- scale visual search and a variety of hashing-based method- s have been proposed Survey paper : Hashing for similarity search . The application of hashing to ...
4
votes
1answer
334 views

Combining multiple variables into one “score”

My question is very similar to this one, which was not solved unfortunately. I am working on a project for which I want to rank countries by means of their HIV/AIDS burden. So I collected a lot of ...
1
vote
0answers
11 views

Label reduction on dataset

This question related to this other one, for which I have devised a strategy and now want some feedback on it. My data consists of 434042 rows, each corresponding to an observation tagged with 1 of ...
4
votes
0answers
47 views

Why would I ever use a linear autoencoder for dimensionality reduction?

Following on from: What're the differences between PCA and autoencoder? If I want to do dimensionality reduction and restrict myself to using a linear activation for my autoencoder, is there any ...
0
votes
0answers
30 views

Reducing number of labels in a dataset

I have a dataset that contains of 13 variables, of which: 5 are binary (T/F). 4 are categorical. 1 is ordinal (ranking). 1 is continuous. 1 is the time component. 1 is a final categorical variable, ...
0
votes
0answers
38 views

Is there a supervised/semi supervised version of pca for dimensionality reduction?

PCA can give me the proper result if "Large variances have important dynamics" holds true for the data. In other words if I want to know along which components the variance of my data is maximized, ...
4
votes
3answers
380 views

random forest - summarize two features in one without losing information

I am training a random forest on a dataset including both categorical and numerical features. In particular I have a binary feature, call it $x_1$, which has $0$ or $1$ as possible outcomes. I also ...
0
votes
0answers
19 views

Understanding filter space in convolutional neural networks and its reduction in Inception architecture

From this source I acquired a quite good understanding of 1x1 convolutions in Inception CNN and how they perform a reduction in the filters dimension. There is one thing I would like to clarify ...
3
votes
0answers
21 views

Data reduction and xgboost(or other boosting and decisision tree methods)

I wonder, does data reduction(ex:factor analysis) have an impact on the result of boosting(ex:xgboost) or decision trees methods other than time gain?
1
vote
0answers
34 views

How much data versus dimensions are needed to overcome the curse of dimensionality?

Are there any guidelines for knowing how many training samples are needed based on the number of features you have in order to not have accuracy degradation as a result of the curse of dimensionality? ...
0
votes
0answers
15 views

Correlation between matrix variables and its PCA scores

Suppose a matrix $X$ and $T$ the score matrix obtained from a PCA decomposition of $X$. Denote as $x_i$ and $t_i$ the columns of $X$ and $T$ respectively. Is there any reason for which $cor(x_i, ...
1
vote
1answer
33 views

How to reduce dimensionality of audio data that comes in form of matrices and vectors?

I'm working on a project involved with identifying different types of sounds (such as screams, singing, and bangs) from each other. We've got our data a reasonable number of different transformations ...
0
votes
0answers
27 views

Kernel PCA increases dimensionality compared with PCA?

I was trying to use sklearn to perform kernel PCA with 28*28 = 784 dims data. At first I used PCA to reduce dimensionality and I chose to reduce to k dimensions where k could explain 95% of the ...
1
vote
0answers
27 views

Handling datasets with categorical variables of many levels

I am working on the Diabetes in 130 US hospitals for years 1999--2008 dataset. After removing unnecessary variables (i.e. some IDs or near-zero-variance variables) and doing some naive imptuation, I ...
0
votes
0answers
36 views

Text Mining Using irlba for dimension reduction

I am trying to do some dimension reduction on a sparse matrix I have. The data is text data currently formatted in a document term matrix. I did some reading and used the irlba package to reduce the ...
3
votes
1answer
61 views

Does it make sense to use PCA when the determinant of the correlation matrix is (almost) zero?

I'm running a PCA over a data set of $N \times p$ size ($N\approx 1000$ being the number of measurements and $p\approx 200$ being the number of dimensions/predictors). I expect many of the predictors ...
0
votes
0answers
35 views

Can someone please explain about squared loading values in ClustOfVar package

I am using ClustOfVar package in R to cluster both nominal and interval variables for dimensionality reduction. Can someone please explain what is squared loading values. how to pick representative ...
1
vote
2answers
74 views

Projecting to lower/higher-dimensional space for classification: dimensionality reduction vs kernel trick

Whilst learning about classification, I have seen two different arguments. One is that projecting the data to a lower-dimensional space, such as with PCA, makes the data more easily separable. The ...
0
votes
0answers
8 views

how to define which of measured, non gaussian variables are effective in discriminate given groups?

I have eight groups (100 samples each) and 43 evaluated variables, with different distributions (some of them are right skewed, some left skewed, some with many zeros etc.). I'm trying to understand ...
0
votes
0answers
77 views

A technique like truncated SVD that uses non-orthogonal components?

Question: Is there a technique like truncated SVD that instead of orthogonal components relies on non-orthogonal components? Specifically, just like a truncated SVD computes simultaneously the ...
2
votes
1answer
109 views

Is each of the PCA or PLS components just one of the original variables?

I am confused about what a component is in PCA and PLS. Are the components just the original variables but not necessarily in the same order? For example, in PCA, if I had 8 variables in my data, ...
2
votes
0answers
47 views

Ranking of neurons of bottleneck layer in AutoEncoder network, similar to components in PCA

When using Principal Component Analysis (PCA) for dimension reduction, the extracted components (basis vectors) are sorted based on the eigenvalues of the covariance matrix of data, i.e. we have the ...
0
votes
0answers
23 views

How exactly is dependent variable expressed in terms of independent variables using Partial Least Square Regression? [duplicate]

I understand the working of NIPALS algorithm but while doing the regression using PLS how exactly the relation between known and unknown is established using Principle Component Analysis. The idea is ...
0
votes
1answer
59 views

LDA for dimensionality reduction usage

I have a original dataset with 70 samples, each sample with 96 features. The samples are labeled as 0 or 1. So I use linear discriminant analysis (LDA) to reduce the dimensionality of all the dataset, ...
0
votes
0answers
14 views

sparse principal component analysis

I have a dataset includes of evaluated KPIs collected from experts. It has 67 dimensions or variables(KPIs) and the number of evaluated data that I have collected is around 50 for each KPIs. I want to ...
1
vote
1answer
19 views

How to remove the effect of other dimensions?

I have data of 3 Dimensions. the value of the third dimension is highly correlated with the value of the first two dimensions. I want to remove the effect of the first two dimensions on the third ...
0
votes
1answer
48 views

Performance of a classifier change heavily

I'm using a data set of 32 face persons and a svm-rbf to classify and a random group of 70% for train and 30% for test. The problem is that my results are heavily dependent of the random group used ...
0
votes
1answer
61 views

Dimensionality reduction for high dimensional sparse data before clustering or spherical k-means?

I am trying to build my first recommender system where i create a user feature space and then cluster them into different groups. Then for the recommendation to work for a particular user , first i ...
3
votes
0answers
28 views

Rules of thumb for a proportion of outliers depending on the dimension

I am implementing and benchmarking different "robust" PCA (principal component analysis, see for instance Robust Principal Component Analysis?) for data that should align (I have no prior on the ...
3
votes
0answers
61 views

How to do dimensionality reduction on a huge data set?

I am working with fMRI data of ~1000 subject. Each subject has a feature vector of ~150 million dimension. So I can only keep the feature vectors of ~10 subjects in memory. What are some algorithms ...
2
votes
1answer
105 views

Meaning of “reconstruction error” in PCA and LDA

I am implementing PCA, LDA, and Naive Bayes, for compression and classification respectively (implementing both an LDA for compression and classification). I have the code written and everything ...
1
vote
1answer
94 views

Can PCA be applied twice or more? [duplicate]

I have a very high-dimensional dataset with 27k features. I want to a reduce the dimensionality of the dataset. I want to use PCA to reduce the dimensionality to 2 as the toolbox I am using expects 2 ...