# Questions tagged [dimensionality-reduction]

Refers to techniques for reducing a large number of variables or dimensions spanned by data to a smaller number of dimensions while preserving as much information about the data as possible. Prominent methods include PCA, MDS, Isomap, etc. The two main subclasses of techniques: feature extraction and feature selection.

1,006 questions
Filter by
Sorted by
Tagged with
1 vote
0 answers
30 views

### Does a CNN always learn a latent space?

In general, a latent space is a structure of reduced dimensionality than that of the input space where points on this space share resemblance the closer they are to each other. This article also ...
• 619
1 vote
0 answers
29 views

### How does UMAP deals with the curse of dimensionality?

This question is related to this one which was not answered. The curse of dimensionality states that in high dimension every distance between pairs of points tends to be the same. See this answer for ...
• 119
0 votes
0 answers
16 views

### How many observations should there be for the less frequent level of a binary variable, in order to include it in MCA?

I am conducting a multiple correspondence analysis (MCA) on several binary variables. This link says: The graphs above can be used to identify variable categories with a very low frequency. These ...
• 1,641
0 votes
0 answers
31 views

### Centering and scaling in Partial Least Squares

I am trying to understand, how data is centered and scaled in Partial Least Squares (PLS). I understand how it is done in Principal Component analysis (PCA). For example, in PCA test-data is centered ...
• 31
1 vote
0 answers
19 views

### Does PCA centering guarantee projection optimality onto an affine set subject to dimensionality constraint?

Say I have $m$ points in $R^n$, not necessarily with sum 0, and I want to project them onto an affine set $S$ with $\dim(S)=d$ for a given $d$ so as to minimize the sum of the squared distances ...
-1 votes
0 answers
15 views

### What are the pre-requisite of a dataset's measurement type before Principal Component Analysis can be used? [duplicate]

Must all the measured data be discrete, continuous, nominal or ordinal? What are some of the transformation techniques that can be used?
• 111
1 vote
0 answers
16 views

### Best way to test whether multidimensional data varies mostly in one direction?

I have some multidimensional data. I would like to measure how "one directional" the data is. For example, the rows in this sample ...
• 11
0 votes
0 answers
50 views

### Sum of PCA principal components

Short I wonder is it possible to sum the principal components together to obtain a score? For example, PC1 + PC2. Details I got the below dataframe: admin_username sales sign book team_sales ...
• 1
1 vote
0 answers
11 views

### Find combination of time-dependent values that shows largest anomaly after event

What I have are timeseries of about 500 variables (which are more or less correlated to each other). The variables refer to standardized anomalies (mean=0, std=1). I now define certain event dates in ...
• 11
0 votes
0 answers
10 views

### Can you class averaging 3D data across the third axis as feature selection / dimensionality reduction?

I am currently doing some research and as part of one of my projects, I had a dataset comprising of approx 200x200x200 data. The research is partly to do with ...
• 11
2 votes
1 answer
56 views

### Will PCA always fit a model at least as well as NMF?

If I perform PCA/NMF on a dataset, and then use the reduced models to reconstruct the original dataset, it seems to me that PCA should typically outperform NMF, simply due to the fact that NMF has the ...
0 votes
0 answers
14 views

### What could cause having more dimensions than variables after MCA and dimensions explaining very little about data?

I have a dataset with 19 variables and 100k observations. All of my variables are categorical, some of them ordinal but I have not taken that into account here. To reduce its dimension, I performed a (...
4 votes
3 answers
164 views

### Curse of dimensionality: How PCA improves my model?

After having read about the curse of dimensionality, I have been looking into a filtered version of the Superconductivity dataset. The number of dimensions is 81, so initially I thought that reducing ...
• 101
1 vote
0 answers
18 views

### How to test the significance of contribution of a variable in Factor analysis and what is the resonable rule for removing a variable from FA?

My question is about the credibility from the statistical point of view of what I have seen in some papers where the researchers (non-statistician) define some kind of staged factor analysis (FA) and ...
0 votes
0 answers
12 views

### Canonical Correlation Analysis CCA dimensionality reduction in R

I want to conduct dimensionality reduction using CCA. Let's suppose that $X$ is a matrix of $p$ predictors, $n$ samples and $y$ is a binary vector of $0$ and $1$, i.e. ...
• 249
8 votes
2 answers
261 views

### PCA as a Cure for the Curse of Dimensionality

I would like some clarification as to how principal component analysis mitigates the Curse of Dimensionality problem. My particular interest is in curbing overfitting in my modelling, or more ...
0 votes
0 answers
8 views

### Is it possible to "warp" samples of different dimensions for sparse PCA?

It's my understanding that for $n$ samples each with $k$ dimensions I can utilize dimensionality reduction methods (ie. PCA, or in my case, sparse PCA) in a straightforward way. But what if my samples ...
• 151
0 votes
0 answers
18 views

### Factor Analysis or PCA on part of or grouped variables

I am working on industrial time series data. (Such as sensor and controller signals, etc.) I have 150 features. Some of my features/indepentdent variables are highly correlated with each other. (75-80-...
• 21
1 vote
2 answers
71 views

### What is the maximum number of dimensions in MDS?

If I have an arbitrary Euclidean distance matrix $D=(d_{ij}:i=1,\ldots,I; j=1,\ldots,I)$ and I want to reconstruct its elements (pairwise Euclidean distances) via classical Euclidean MDS. That is find ...
• 402
0 votes
0 answers
21 views

### On assumptions and calculation of O'Brien's "Global Statistical Test" for Multiple Endpoints Analysis

As I understand, O'Brien's Global Statistical Test (GST) (https://pubmed.ncbi.nlm.nih.gov/6534410/) for Multiple Endpoints Analysis is a method for mapping a multivariate problem on a univariate scale....
• 63
0 votes
1 answer
32 views

### PCA and LDA segregate the classes perfectly in 3 dimensions

I'm applying PCA and LDA on my gene expression microarray data to reduce dimensionality. It has ~20,500 features. I tried to visualize the first 3 components from both the methods in a 3d scatter plot ...
2 votes
0 answers
69 views

### Why do I get different results for the Isolation Forest after using PCA?

I am working on an unsupervised anomaly detection project and used the Isolation Forest and AutoEncoders (a normal one and a VAE) to detect anomalies. The AutoEncoders' prediction are identical but ...
1 vote
0 answers
18 views

### Are multitable analyses like Multifactor Analysis (MFA), STATIS, and Partial Triadic Analysis (PTA) JUST exploratory methods? Can I use PC Scores?

If I perform PCA on a simple table, I can take the resulting principal component scores as variables and then perform regression to predict an outcome from my original data. I would do this for ...
0 votes
1 answer
33 views

### Variance used in t-SNE

I was going through the t-SNE, but I am bit confused. While the original paper of t-SNE is based on the SNE and SNE uses $\sigma_i^2$ (note the subscript $i$) while calculating the similarity of point ...
4 votes
4 answers
1k views

### How can I reduce a very large sample size for statistical significance (sampling methods)? [duplicate]

In my biological study, I have around 14000 independent samples, and I study the evolution of a response variable over time. I have three groups to study. Thus, I have two factors: factor "Group&...
• 457
1 vote
0 answers
104 views

### Distance metric that is robust to collinearity

I'm trying to find a distance metric that takes into account the correlation between vectors. That is, suppose we have matrix $M$ of dimensions $n \times k$, and we take the pairwise distance between ...
• 231
0 votes
0 answers
22 views

### Using dimensionality reduction to control for more variables in a multiple linear regression without breaking the 15:1 rule - possible?

I'm interested in whether there is a relationship between my particular variable of interest (X) and my dependent variable (Y). I'm planning to explore this using multiple linear regression. My focus ...
• 11
1 vote
1 answer
38 views

### Is there an incremental dimensionality reduction algorithm that can handle batch size less than number of components to be reduced?

I have a large dataset of patient data by hour. For example, given the shape as (hours, features), patient 1 data shape could be ...
• 123
3 votes
0 answers
47 views

### How many features/dimensions/variables can K-Nearest Neighbors handle with a finite amount of data?

KNN is an algorithm where the "Curse of Dimensionality" applies extremely literally and directly. Let's take some kind of basic, 50/50 balanced, binary classification problem. I'm wondering ...
• 1,082
0 votes
0 answers
30 views

### Interpreting Sparse PCA components

I am reading about Sparse PCA and I am experimenting with a dataset that consists of 9 feature variables, I am using Python Sklearn and the results I get when checking the components_ attributes is ...
0 votes
1 answer
27 views

### Practical significance of number of singular values in SVD

I am working on a binary classification problem. SVD is used for dimensionality reduction and the vector with reduced dimension is used as the feature vector. DNN is used as the classifier. There are ...
• 11
1 vote
0 answers
23 views

### Under what conditions does univariate variable prescreening fail? (random forest modeling)

Univariate statistical methods are often used to prescreen variables for possible inclusion in a model. For example, running random forests (RF) with only a single predictor at a time to prescreen ...
• 273
0 votes
0 answers
27 views

### Does using LDA components in classification models lead to feature leakage

I have a multiclass classification problem with (too) many independent variables. And in order to reduce the number of features I did a Linear Discriminant Analysis (LDA) on my data, which created two ...
0 votes
0 answers
40 views

### Using Partial Least Squares for reduced-dimension machine learning

I want to perform dimensionality reduction using Partial Least Squares on a complex, large-dimension data set before training various regression models on the reduced-dimension data set. I understand ...
• 167
0 votes
0 answers
10 views

### Combining multiple datasets vs multiple models in high dimensions

This question is related to this one and this one, but I was wondering about this topic in general. Imagine a setting where multiple datasets, representing different measurements, have been gathered. ...
• 514
0 votes
0 answers
17 views

### Is there any methods to identify when PCA transformation is not optimal

I am working on unsupervized clustering problematic. I am currently reducing the dimensions of my data by applying first a PCA transformation through which I keep 80% of the variance (reducing ...
• 235
0 votes
0 answers
44 views

### Dimension reduction with categorical and quantitative data. (up to a million data entries)

I have data on T cell receptor gene usage: [1]: https://i.stack.imgur.com/z7Jj8.png Essentially this is categorical data (first 4 columns) with a quantitative frequency column. I have tried using PCA ...
1 vote
0 answers
65 views

### How is UMAP a valid dimensionality reduction technique when it uses KNN, which suffers from the curse of dimensionality?

I have not a found a satisfactory, or really any answer, to the following problem that I cannot resolve myself. UMAP is touted as an excellent dimensionality reduction technique by constructing a high-...
• 104
0 votes
0 answers
19 views

### Using UMAP on the space of principal components, a valid proposition?

I have a genomics dataset with roughly 16,000 features. Currently, I'm in the process of clustering for cell subtype identification, which I'll then build a classifier on. However, I've run into two ...
• 104
0 votes
0 answers
17 views

### Finding independent variables from tabulated dataset

Given a tabulated dataset for $n$ variables, how can I find the smallest subset that consists of $m$ independent variables ($m \leq n$), so that the complete dataset can be constructed by knowing only ...
• 101
0 votes
0 answers
24 views

### How to choose the best ICA function

I am using the fastICA library from scikit-learn on a project and noticed that there was a fun parameter, referencing the ...
• 235
0 votes
1 answer
46 views

### Transforming input features for t-SNE and UMAP

Previously I was using PCA as my dimension reduction algorithm of choice, but have recently moved to using t-SNE and UMAP. For PCA I would apply transformations to my input features to ensure they ...
• 101
1 vote
1 answer
175 views

### Reconstruction error in PCA?

I'm using PCA for a while, but recently I read about reconstruction error which I cannot understand... For example let's consider dataset consisting 5 variables: $X_1, X_2, X_3, X_4, X_5$. ...
• 289
1 vote
1 answer
67 views

### Find first Principal Component (and loading) using a fast iterative algorithm without covariance matrix

I have a matrix $X$ and I would like to find its first principal component and the corresponding loadings. I would like to do this without computing the covariance matrix of $X$. How can I do so? This ...
2 votes
1 answer
89 views

### Principal component analysis of RT-qPCR data

Greetings to all biostatisticians, I am analyzing a gene expression data set consisting of around 100 genes that were measured by RT-qPCR and expession values are given as 2^-delta Ct. Expression of ...
3 votes
1 answer
58 views

### Is Nonnegative matrix factorization a clustering method or a dimensionality reduction method?

In the matrix factorization we have the problem of decomposing a nonnegative matrix $X$ into two lower-rank matrices $W$ and $H$. I would like to know whether this method is considered as a dimension ...
• 143
0 votes
0 answers
100 views

### What does it mean if we use eigenvetor matrix of centered matrix $\mathbf{X}$ to reduce the dimension of $\mathbf{X}$ without centering?

Recently, I was trying to use PCA to reduce the dimension of matrix $\mathbf{X}$. Suppose a data matrix $\mathbf{X}\in \mathbb{C}^{M\times N}$, where $M$ is the number of variables and $N$ is the ...
0 votes
0 answers
14 views

### How can I use projections (components) from multiple correspondence analysis in subsequent regression analysis, similar to PCA

I am trying to reduce the dimension of a matrix of several hundred binary/indicator/boolean variables, and then use the reduced components in subsequent regression modeling. For continuous variables, ...
• 25
0 votes
0 answers
36 views

### What effect would clusters get when adding more variables to clustering task?

I did kmeans++ clustering for 100 clusters on user data. When I first tried clustering with two variables, I set the number of clusters to 100 and looked at the result of clustering. The number of ...
• 2,583
0 votes
0 answers
12 views

### Can regularization be used when your features >> number of observations to reduce the feature space?

I'm wondering if a reasonable way of reducing the feature space when p >> n is to simply use l1/l2 regularization. Will this work? Or can the model simply not be fit to begin with, and so the ...