Refers to techniques for reducing a large number of variables to a smaller number while preserving as much information as possible. Prominent methods include PCA, MDS, Isomap, etc.

learn more… | top users | synonyms

1
vote
0answers
19 views

Dimensionality reduction for multivariate time series

I have a data set including 25 variables $(x_{1,t},\dotsc,x_{25,t})$ at each time $t$ and all of this group is repeated through time. I want to explore the relationship between these variables through ...
0
votes
0answers
14 views

How can I have more factors than there are variables

I've come to learn about factor analysis as mainly a dimensionality reduction tool. However, can we also use the factor analysis method to have more factors than there are variables? If so, what would ...
0
votes
0answers
8 views

Problem with syntetic data generating for Probabilistic PCA and Factor Analysis (FA) comparison - methodology

I am trying to understand a short example related to dimension reduction from python scikit-learn.org official documentation for long time and unfortunately I am not successful. I don't have problems ...
-3
votes
0answers
31 views

How do i use PCA? [duplicate]

I am not sure i understood the concept of PCA that well. I know it is used to reduce the dimensionality by the help of the eigenvector and eigenvalues of the covariance matrix, and thereby compute ...
12
votes
4answers
2k views

Does “curse of dimensionality” really exist in real data?

I understand what is "curse of dimensionality", and I have done some high dimensional optimization problems and know the challenge of the exponential possibilities. However, I doubt if the "curse of ...
0
votes
1answer
72 views

Partitioning training data for dimension reduction and classification

Let's say I want to test the performance of my dimension reduction + classification pipeline. To do this, I will use k-fold cross validation. I know that performing dimension reduction on the complete ...
0
votes
0answers
25 views

Does it make sense to do PCA before kernel regression?

I have a set of features extracted from the same samples and I'm learning a kernel ridge regression. Now, especially for feature fusion, reducing the number of features before combining them seems ...
0
votes
0answers
34 views

Variable selection in logistic regression model

Imagine a data set with approximately 100 variables and 5000 cases. The outcome is a two-level factor. All variables are factors, most of them three levels (yes, no, or indifferent). After building a ...
2
votes
3answers
97 views

Autoencoder wrongly removes objects from images

For a university project, we want to use reinforcement learning from raw camera input to teach a robot to hit a ball. It works when we detect the ball and feed that to the algorithm, so raw data is ...
0
votes
0answers
4 views

Embedding categorical values

Is it possible to perform embedding of a dataset that contains only categorical values e.g. n f1 f2 f3 1 2 1 3 2 2 2 3 3 1 3 2 4 1 4 1 where n is an object and f1, f2, f3 are ...
0
votes
0answers
7 views

Identification in a gaussian two factor model

I am working with a Gaussian two factor model: $$ X_i = \beta_iZ_1+\gamma_iZ_2+\varepsilon_i, \space i = 1,2,...,n $$ where $Z_j\sim N(0,1), \space Z_1 \perp Z_2 $ and $\varepsilon_i\space iid\space ...
1
vote
0answers
7 views

How flexible are autoencoders for non-linear dimensionality reduction?

I've been starting to play around with autoencoders for feature extraction and dimensionality reduction, and am wondering how critical input feature definitions are for success. For example, some of ...
0
votes
0answers
34 views

Can Factor Analysis on Mixed Data be treated like a PCA?

I wanted to do something equivalent to a PCA on a mixed data set containing categorical variables and continuous numerical predictor variables which are normally distributed but measured in very ...
1
vote
0answers
19 views

Function Separation

I have a lot of measurements of a function f(x,y) (lets say a lot more than a million measurements) at known values of the vectors x and y. The inputs to the function are the vectors x and y and the ...
2
votes
3answers
57 views

2 features and 2 principal components

Whats the difference between 2 features and 2 principal components? I know what a PCA is, I just have the following problem: If my data has 2 features, the PCA will produce 2 components. So why does ...
-1
votes
1answer
37 views

When can clustering be used for dimensionality reduction? [closed]

Can a clustering method be used for dimensionality reduction? I though the answer would be that the cluster numbers can act as the synthetic reduced dimension -- but the other day a friend had a more ...
0
votes
1answer
140 views

PCA vs FA vs ICA for dimensionality reduction in questionaire data

I am trying to identify personality traits underlying the multidimensional data from a questionnaire. In more abstract terms this means reducing the dimensionality of my data from N-dimensional (where ...
3
votes
1answer
72 views

Do I need to run PCA over all predictors in a regression model? Can I run it only over the continuous ones?

I'm looking at the Lending Club data from Kaggle and I'm just building a pretty simple model to predict defaults. The data has a large amount of both continuous and categorical variables (I have ...
1
vote
0answers
37 views

How to determine time complexity of EM algorithm of probabilistic PCA?

I was studying probabilistic PCA from Bishop's book. There an EM algorithm is provided to calculate principal subspace: Here $\mathbf M$ is $M\times M$ matrix, $\mathbf W$ is $D\times M$ matrix ...
2
votes
0answers
70 views

If almost all variance is explained by the few first principal components, what can we say about the dataset?

What can we say about a dataset if we apply PCA and observe that there is a high percentage of variance in the first principal component(s)? Can we say that this dataset has linear structure? Can we ...
1
vote
1answer
35 views

How does one visualize the self-organizing map of $n$-dimensional data

I have a data set consisting from $7$-dimensional data points. I want to produce a self-organizing map for this data to see how my data is clustered. I have been reading some tutorials from the ...
0
votes
1answer
96 views

Why convert categorical data into numerical using one hot encoding

I don't have very strong statistical background, and I'm new in data science... Now, I am practicing PCA (Principle Component Analysis) for dimension reduction. This tutorial looks very complete, but ...
6
votes
1answer
168 views

Is PCA a non-linear transform?

In the article Relative Information Loss in the PCA, the authors make, at some point (in the introductory section), the following statement: In case the orthogonal matrix is not known a priori, ...
0
votes
1answer
47 views

Negative coefficients for ordinal logistic regression in R

I am trying to model my dependent variable (ordinal - three levels) using a set of independent variables (5 ordinal and 10 numeric). I am using lrm function in "rms"...
1
vote
0answers
9 views

Feature extraction from data in the form of many manifolds, in hierarchial structure and various dimensions

Is there a known feature extraction method which was developed to cope with data that satisfies the following assumptions?: The data points are real valued vectors in ...
1
vote
0answers
17 views

How to choose an optimal dimension reduction factor in LSA processing

I'm performing a K-Means clustering on a 400.000 text dataset. After eliminating useless chars and removing stopwords, I get a dictionnary size of around 42000 words. So before doing the clustering, ...
0
votes
0answers
12 views

Filter Feature Selection approaches for continuous variables?

I've noticed that correlation-based filtering for selecting features in high dimensional data require discretization of continuous variables, like e.g. Fast Correlation-based Filtering or regular CFS. ...
9
votes
1answer
171 views

What is the connection between partial least squares, reduced rank regression, and principal component regression?

Are reduced rank regression and principal component regression just special cases of partial least squares? This tutorial (Page 6, "Comparison of Objectives") states that when we do partial least ...
2
votes
1answer
27 views

Self Organizing Maps: How is the location computed and updated?

I have read other similar questions on here, but I am still unsure how SOM deals with the positions/locations of the neurons. Say that the input space is N-dimensional. I initalise some weights, and ...
0
votes
0answers
23 views

PCA + Parallel Analysis

When I realize the Factor Analysis (I have 16 items), the PCA says I have 5 factors. But in the scree plot there is no elbow at all, just a decreasing line, that makes me think maybe I shouldn't be ...
1
vote
0answers
10 views

Understanding the bound given by Johnson–Lindenstrauss lemma

This was a question I asked in the mathematics site. Though no one replied but I have figured it out. But new questions arise. Former Question: Here I choose to use the statement made by S.Dasgupta: ...
0
votes
0answers
21 views

Help in understanding hashing for nearest neighbor search

Hashing is a technique for large- scale visual search and a variety of hashing-based method- s have been proposed Survey paper : Hashing for similarity search . The application of hashing to ...
4
votes
1answer
343 views

Combining multiple variables into one “score”

My question is very similar to this one, which was not solved unfortunately. I am working on a project for which I want to rank countries by means of their HIV/AIDS burden. So I collected a lot of ...
1
vote
0answers
11 views

Label reduction on dataset

This question related to this other one, for which I have devised a strategy and now want some feedback on it. My data consists of 434042 rows, each corresponding to an observation tagged with 1 of ...
4
votes
0answers
58 views

Why would I ever use a linear autoencoder for dimensionality reduction?

Following on from: What're the differences between PCA and autoencoder? If I want to do dimensionality reduction and restrict myself to using a linear activation for my autoencoder, is there any ...
0
votes
0answers
30 views

Reducing number of labels in a dataset

I have a dataset that contains of 13 variables, of which: 5 are binary (T/F). 4 are categorical. 1 is ordinal (ranking). 1 is continuous. 1 is the time component. 1 is a final categorical variable, ...
0
votes
0answers
42 views

Is there a supervised/semi supervised version of pca for dimensionality reduction?

PCA can give me the proper result if "Large variances have important dynamics" holds true for the data. In other words if I want to know along which components the variance of my data is maximized, ...
4
votes
3answers
383 views

random forest - summarize two features in one without losing information

I am training a random forest on a dataset including both categorical and numerical features. In particular I have a binary feature, call it $x_1$, which has $0$ or $1$ as possible outcomes. I also ...
0
votes
0answers
26 views

Understanding filter space in convolutional neural networks and its reduction in Inception architecture

From this source I acquired a quite good understanding of 1x1 convolutions in Inception CNN and how they perform a reduction in the filters dimension. There is one thing I would like to clarify ...
3
votes
0answers
26 views

Data reduction and xgboost(or other boosting and decisision tree methods)

I wonder, does data reduction(ex:factor analysis) have an impact on the result of boosting(ex:xgboost) or decision trees methods other than time gain?
1
vote
0answers
38 views

How much data versus dimensions are needed to overcome the curse of dimensionality?

Are there any guidelines for knowing how many training samples are needed based on the number of features you have in order to not have accuracy degradation as a result of the curse of dimensionality? ...
0
votes
0answers
15 views

Correlation between matrix variables and its PCA scores

Suppose a matrix $X$ and $T$ the score matrix obtained from a PCA decomposition of $X$. Denote as $x_i$ and $t_i$ the columns of $X$ and $T$ respectively. Is there any reason for which $cor(x_i, ...
1
vote
1answer
35 views

How to reduce dimensionality of audio data that comes in form of matrices and vectors?

I'm working on a project involved with identifying different types of sounds (such as screams, singing, and bangs) from each other. We've got our data a reasonable number of different transformations ...
0
votes
0answers
29 views

Kernel PCA increases dimensionality compared with PCA?

I was trying to use sklearn to perform kernel PCA with 28*28 = 784 dims data. At first I used PCA to reduce dimensionality and I chose to reduce to k dimensions where k could explain 95% of the ...
2
votes
0answers
34 views

Handling datasets with categorical variables of many levels

I am working on the Diabetes in 130 US hospitals for years 1999--2008 dataset. After removing unnecessary variables (i.e. some IDs or near-zero-variance variables) and doing some naive imptuation, I ...
0
votes
0answers
41 views

Text Mining Using irlba for dimension reduction

I am trying to do some dimension reduction on a sparse matrix I have. The data is text data currently formatted in a document term matrix. I did some reading and used the irlba package to reduce the ...
3
votes
1answer
64 views

Does it make sense to use PCA when the determinant of the correlation matrix is (almost) zero?

I'm running a PCA over a data set of $N \times p$ size ($N\approx 1000$ being the number of measurements and $p\approx 200$ being the number of dimensions/predictors). I expect many of the predictors ...
0
votes
0answers
41 views

Can someone please explain about squared loading values in ClustOfVar package

I am using ClustOfVar package in R to cluster both nominal and interval variables for dimensionality reduction. Can someone please explain what is squared loading values. how to pick representative ...
1
vote
2answers
87 views

Projecting to lower/higher-dimensional space for classification: dimensionality reduction vs kernel trick

Whilst learning about classification, I have seen two different arguments. One is that projecting the data to a lower-dimensional space, such as with PCA, makes the data more easily separable. The ...
0
votes
0answers
8 views

how to define which of measured, non gaussian variables are effective in discriminate given groups?

I have eight groups (100 samples each) and 43 evaluated variables, with different distributions (some of them are right skewed, some left skewed, some with many zeros etc.). I'm trying to understand ...