Questions tagged [dimensionality-reduction]
Refers to techniques for reducing a large number of variables or dimensions spanned by data to a smaller number of dimensions while preserving as much information about the data as possible. Prominent methods include PCA, MDS, Isomap, etc. The two main subclasses of techniques: feature extraction and feature selection.
1,006
questions
1
vote
0
answers
30
views
Does a CNN always learn a latent space?
In general, a latent space is a structure of reduced dimensionality than that of the input space where points on this space share resemblance the closer they are to each other.
This article also ...
- 619
1
vote
0
answers
29
views
How does UMAP deals with the curse of dimensionality?
This question is related to this one which was not answered.
The curse of dimensionality states that in high dimension every distance between pairs of points tends to be the same. See this answer for ...
- 119
0
votes
0
answers
16
views
How many observations should there be for the less frequent level of a binary variable, in order to include it in MCA?
I am conducting a multiple correspondence analysis (MCA) on several binary variables.
This link says:
The graphs above can be used to identify variable categories with a
very low frequency. These ...
- 1,641
0
votes
0
answers
31
views
Centering and scaling in Partial Least Squares
I am trying to understand, how data is centered and scaled in Partial Least Squares (PLS). I understand how it is done in Principal Component analysis (PCA).
For example, in PCA test-data is centered ...
- 31
1
vote
0
answers
19
views
Does PCA centering guarantee projection optimality onto an affine set subject to dimensionality constraint?
Say I have $m$ points in $R^n$, not necessarily with sum 0, and I want to project them onto an affine set $S$ with $\dim(S)=d$ for a given $d$ so as to minimize the sum of the squared distances ...
-1
votes
0
answers
15
views
What are the pre-requisite of a dataset's measurement type before Principal Component Analysis can be used? [duplicate]
Must all the measured data be discrete, continuous, nominal or ordinal? What are some of the transformation techniques that can be used?
- 111
1
vote
0
answers
16
views
Best way to test whether multidimensional data varies mostly in one direction?
I have some multidimensional data. I would like to measure how "one directional" the data is. For example, the rows in this sample
...
- 11
0
votes
0
answers
50
views
Sum of PCA principal components
Short
I wonder is it possible to sum the principal components together to obtain a score? For example, PC1 + PC2.
Details
I got the below dataframe:
admin_username
sales
sign
book
team_sales
...
- 1
1
vote
0
answers
11
views
Find combination of time-dependent values that shows largest anomaly after event
What I have are timeseries of about 500 variables (which are more or less correlated to each other). The variables refer to standardized anomalies (mean=0, std=1). I now define certain event dates in ...
- 11
0
votes
0
answers
10
views
Can you class averaging 3D data across the third axis as feature selection / dimensionality reduction?
I am currently doing some research and as part of one of my projects, I had a dataset comprising of approx 200x200x200 data.
The research is partly to do with ...
- 11
2
votes
1
answer
56
views
Will PCA always fit a model at least as well as NMF?
If I perform PCA/NMF on a dataset, and then use the reduced models to reconstruct the original dataset, it seems to me that PCA should typically outperform NMF, simply due to the fact that NMF has the ...
0
votes
0
answers
14
views
What could cause having more dimensions than variables after MCA and dimensions explaining very little about data?
I have a dataset with 19 variables and 100k observations. All of my variables are categorical, some of them ordinal but I have not taken that into account here. To reduce its dimension, I performed a (...
4
votes
3
answers
164
views
Curse of dimensionality: How PCA improves my model?
After having read about the curse of dimensionality, I have been looking into a filtered version of the Superconductivity dataset.
The number of dimensions is 81, so initially I thought that reducing ...
- 101
1
vote
0
answers
18
views
How to test the significance of contribution of a variable in Factor analysis and what is the resonable rule for removing a variable from FA?
My question is about the credibility from the statistical point of view of what I have seen in some papers where the researchers (non-statistician) define some kind of staged factor analysis (FA) and ...
- 11
0
votes
0
answers
12
views
Canonical Correlation Analysis CCA dimensionality reduction in R
I want to conduct dimensionality reduction using CCA. Let's suppose that $X$ is a matrix of $p$ predictors, $n$ samples and $y$ is a binary vector of $0$ and $1$, i.e.
...
- 249
8
votes
2
answers
261
views
PCA as a Cure for the Curse of Dimensionality
I would like some clarification as to how principal component analysis mitigates the Curse of Dimensionality problem. My particular interest is in curbing overfitting in my modelling, or more ...
0
votes
0
answers
8
views
Is it possible to "warp" samples of different dimensions for sparse PCA?
It's my understanding that for $n$ samples each with $k$ dimensions I can utilize dimensionality reduction methods (ie. PCA, or in my case, sparse PCA) in a straightforward way. But what if my samples ...
- 151
0
votes
0
answers
18
views
Factor Analysis or PCA on part of or grouped variables
I am working on industrial time series data. (Such as sensor and controller signals, etc.) I have 150 features. Some of my features/indepentdent variables are highly correlated with each other. (75-80-...
- 21
1
vote
2
answers
71
views
What is the maximum number of dimensions in MDS?
If I have an arbitrary Euclidean distance matrix $D=(d_{ij}:i=1,\ldots,I; j=1,\ldots,I)$ and I want to reconstruct its elements (pairwise Euclidean distances) via classical Euclidean MDS. That is find ...
- 402
0
votes
0
answers
21
views
On assumptions and calculation of O'Brien's "Global Statistical Test" for Multiple Endpoints Analysis
As I understand, O'Brien's Global Statistical Test (GST) (https://pubmed.ncbi.nlm.nih.gov/6534410/) for Multiple Endpoints Analysis is a method for mapping a multivariate problem on a univariate scale....
- 63
0
votes
1
answer
32
views
PCA and LDA segregate the classes perfectly in 3 dimensions
I'm applying PCA and LDA on my gene expression microarray data to reduce dimensionality. It has ~20,500 features. I tried to visualize the first 3 components from both the methods in a 3d scatter plot ...
2
votes
0
answers
69
views
Why do I get different results for the Isolation Forest after using PCA?
I am working on an unsupervised anomaly detection project and used the Isolation Forest and AutoEncoders (a normal one and a VAE) to detect anomalies. The AutoEncoders' prediction are identical but ...
- 31
1
vote
0
answers
18
views
Are multitable analyses like Multifactor Analysis (MFA), STATIS, and Partial Triadic Analysis (PTA) JUST exploratory methods? Can I use PC Scores?
If I perform PCA on a simple table, I can take the resulting principal component scores as variables and then perform regression to predict an outcome from my original data. I would do this for ...
0
votes
1
answer
33
views
Variance used in t-SNE
I was going through the t-SNE, but I am bit confused. While the original paper of t-SNE is based on the SNE and SNE uses $\sigma_i^2$ (note the subscript $i$) while calculating the similarity of point ...
4
votes
4
answers
1k
views
How can I reduce a very large sample size for statistical significance (sampling methods)? [duplicate]
In my biological study, I have around 14000 independent samples, and I study the evolution of a response variable over time. I have three groups to study.
Thus, I have two factors: factor "Group&...
- 457
1
vote
0
answers
104
views
Distance metric that is robust to collinearity
I'm trying to find a distance metric that takes into account the correlation between vectors. That is, suppose we have matrix $M$ of dimensions $n \times k$, and we take the pairwise distance between ...
- 231
0
votes
0
answers
22
views
Using dimensionality reduction to control for more variables in a multiple linear regression without breaking the 15:1 rule - possible?
I'm interested in whether there is a relationship between my particular variable of interest (X) and my dependent variable (Y). I'm planning to explore this using multiple linear regression. My focus ...
- 11
1
vote
1
answer
38
views
Is there an incremental dimensionality reduction algorithm that can handle batch size less than number of components to be reduced?
I have a large dataset of patient data by hour. For example, given the shape as (hours, features), patient 1 data shape could be ...
- 123
3
votes
0
answers
47
views
How many features/dimensions/variables can K-Nearest Neighbors handle with a finite amount of data?
KNN is an algorithm where the "Curse of Dimensionality" applies extremely literally and directly.
Let's take some kind of basic, 50/50 balanced, binary classification problem. I'm wondering ...
- 1,082
0
votes
0
answers
30
views
Interpreting Sparse PCA components
I am reading about Sparse PCA and I am experimenting with a dataset that consists of 9 feature variables, I am using Python Sklearn and the results I get when checking the components_ attributes is ...
- 1
0
votes
1
answer
27
views
Practical significance of number of singular values in SVD
I am working on a binary classification problem. SVD is used for dimensionality reduction and the vector with reduced dimension is used as the feature vector. DNN is used as the classifier.
There are ...
- 11
1
vote
0
answers
23
views
Under what conditions does univariate variable prescreening fail? (random forest modeling)
Univariate statistical methods are often used to prescreen variables for possible inclusion in a model. For example, running random forests (RF) with only a single predictor at a time to prescreen ...
- 273
0
votes
0
answers
27
views
Does using LDA components in classification models lead to feature leakage
I have a multiclass classification problem with (too) many independent variables. And in order to reduce the number of features I did a Linear Discriminant Analysis (LDA) on my data, which created two ...
- 1
0
votes
0
answers
40
views
Using Partial Least Squares for reduced-dimension machine learning
I want to perform dimensionality reduction using Partial Least Squares on a complex, large-dimension data set before training various regression models on the reduced-dimension data set. I understand ...
- 167
0
votes
0
answers
10
views
Combining multiple datasets vs multiple models in high dimensions
This question is related to this one and this one, but I was wondering about this topic in general.
Imagine a setting where multiple datasets, representing different measurements, have been gathered. ...
- 514
0
votes
0
answers
17
views
Is there any methods to identify when PCA transformation is not optimal
I am working on unsupervized clustering problematic.
I am currently reducing the dimensions of my data by applying first a PCA transformation through which I keep 80% of the variance (reducing ...
- 235
0
votes
0
answers
44
views
Dimension reduction with categorical and quantitative data. (up to a million data entries)
I have data on T cell receptor gene usage:
[1]: https://i.stack.imgur.com/z7Jj8.png
Essentially this is categorical data (first 4 columns) with a quantitative frequency column.
I have tried using PCA ...
1
vote
0
answers
65
views
How is UMAP a valid dimensionality reduction technique when it uses KNN, which suffers from the curse of dimensionality?
I have not a found a satisfactory, or really any answer, to the following problem that I cannot resolve myself.
UMAP is touted as an excellent dimensionality reduction technique by constructing a high-...
- 104
0
votes
0
answers
19
views
Using UMAP on the space of principal components, a valid proposition?
I have a genomics dataset with roughly 16,000 features. Currently, I'm in the process of clustering for cell subtype identification, which I'll then build a classifier on. However, I've run into two ...
- 104
0
votes
0
answers
17
views
Finding independent variables from tabulated dataset
Given a tabulated dataset for $n$ variables, how can I find the smallest subset that consists of $m$ independent variables ($m \leq n$), so that the complete dataset can be constructed by knowing only ...
- 101
0
votes
0
answers
24
views
How to choose the best ICA function
I am using the fastICA library from scikit-learn on a project and noticed that there was a fun parameter, referencing the ...
- 235
0
votes
1
answer
46
views
Transforming input features for t-SNE and UMAP
Previously I was using PCA as my dimension reduction algorithm of choice, but have recently moved to using t-SNE and UMAP.
For PCA I would apply transformations to my input features to ensure they ...
- 101
1
vote
1
answer
175
views
Reconstruction error in PCA?
I'm using PCA for a while, but recently I read about reconstruction error which I cannot understand...
For example let's consider dataset consisting 5 variables: $X_1, X_2, X_3, X_4, X_5$. ...
- 289
1
vote
1
answer
67
views
Find first Principal Component (and loading) using a fast iterative algorithm without covariance matrix
I have a matrix $X$ and I would like to find its first principal component and the corresponding loadings. I would like to do this without computing the covariance matrix of $X$. How can I do so?
This ...
- 489
2
votes
1
answer
89
views
Principal component analysis of RT-qPCR data
Greetings to all biostatisticians,
I am analyzing a gene expression data set consisting of around 100 genes that were measured by RT-qPCR and expession values are given as 2^-delta Ct. Expression of ...
3
votes
1
answer
58
views
Is Nonnegative matrix factorization a clustering method or a dimensionality reduction method?
In the matrix factorization we have the problem of decomposing a nonnegative matrix $X$ into two lower-rank matrices $W$ and $H$. I would like to know whether this method is considered as a dimension ...
- 143
0
votes
0
answers
100
views
What does it mean if we use eigenvetor matrix of centered matrix $\mathbf{X}$ to reduce the dimension of $\mathbf{X}$ without centering?
Recently, I was trying to use PCA to reduce the dimension of matrix $\mathbf{X}$.
Suppose a data matrix $\mathbf{X}\in \mathbb{C}^{M\times N}$, where $M$ is the number of variables and $N$ is the ...
- 1
0
votes
0
answers
14
views
How can I use projections (components) from multiple correspondence analysis in subsequent regression analysis, similar to PCA
I am trying to reduce the dimension of a matrix of several hundred binary/indicator/boolean variables, and then use the reduced components in subsequent regression modeling.
For continuous variables, ...
- 25
0
votes
0
answers
36
views
What effect would clusters get when adding more variables to clustering task?
I did kmeans++ clustering for 100 clusters on user data. When I first tried clustering with two variables, I set the number of clusters to 100 and looked at the result of clustering. The number of ...
- 2,583
0
votes
0
answers
12
views
Can regularization be used when your features >> number of observations to reduce the feature space?
I'm wondering if a reasonable way of reducing the feature space when p >> n is to simply use l1/l2 regularization.
Will this work? Or can the model simply not be fit to begin with, and so the ...
