Questions tagged [dimensionality-reduction]

Refers to techniques for reducing a large number of variables or dimensions spanned by data to a smaller number of dimensions while preserving as much information about the data as possible. Prominent methods include PCA, MDS, Isomap, etc. The two main subclasses of techniques: feature extraction and feature selection.

Filter by
Sorted by
Tagged with
1 vote
0 answers
30 views

Does a CNN always learn a latent space?

In general, a latent space is a structure of reduced dimensionality than that of the input space where points on this space share resemblance the closer they are to each other. This article also ...
user avatar
  • 619
1 vote
0 answers
29 views

How does UMAP deals with the curse of dimensionality?

This question is related to this one which was not answered. The curse of dimensionality states that in high dimension every distance between pairs of points tends to be the same. See this answer for ...
user avatar
  • 119
0 votes
0 answers
16 views

How many observations should there be for the less frequent level of a binary variable, in order to include it in MCA?

I am conducting a multiple correspondence analysis (MCA) on several binary variables. This link says: The graphs above can be used to identify variable categories with a very low frequency. These ...
user avatar
  • 1,641
0 votes
0 answers
31 views

Centering and scaling in Partial Least Squares

I am trying to understand, how data is centered and scaled in Partial Least Squares (PLS). I understand how it is done in Principal Component analysis (PCA). For example, in PCA test-data is centered ...
user avatar
  • 31
1 vote
0 answers
19 views

Does PCA centering guarantee projection optimality onto an affine set subject to dimensionality constraint?

Say I have $m$ points in $R^n$, not necessarily with sum 0, and I want to project them onto an affine set $S$ with $\dim(S)=d$ for a given $d$ so as to minimize the sum of the squared distances ...
user avatar
-1 votes
0 answers
15 views

What are the pre-requisite of a dataset's measurement type before Principal Component Analysis can be used? [duplicate]

Must all the measured data be discrete, continuous, nominal or ordinal? What are some of the transformation techniques that can be used?
user avatar
1 vote
0 answers
16 views

Best way to test whether multidimensional data varies mostly in one direction?

I have some multidimensional data. I would like to measure how "one directional" the data is. For example, the rows in this sample ...
user avatar
  • 11
0 votes
0 answers
50 views

Sum of PCA principal components

Short I wonder is it possible to sum the principal components together to obtain a score? For example, PC1 + PC2. Details I got the below dataframe: admin_username sales sign book team_sales ...
user avatar
  • 1
1 vote
0 answers
11 views

Find combination of time-dependent values that shows largest anomaly after event

What I have are timeseries of about 500 variables (which are more or less correlated to each other). The variables refer to standardized anomalies (mean=0, std=1). I now define certain event dates in ...
user avatar
  • 11
0 votes
0 answers
10 views

Can you class averaging 3D data across the third axis as feature selection / dimensionality reduction?

I am currently doing some research and as part of one of my projects, I had a dataset comprising of approx 200x200x200 data. The research is partly to do with ...
user avatar
2 votes
1 answer
56 views

Will PCA always fit a model at least as well as NMF?

If I perform PCA/NMF on a dataset, and then use the reduced models to reconstruct the original dataset, it seems to me that PCA should typically outperform NMF, simply due to the fact that NMF has the ...
user avatar
0 votes
0 answers
14 views

What could cause having more dimensions than variables after MCA and dimensions explaining very little about data?

I have a dataset with 19 variables and 100k observations. All of my variables are categorical, some of them ordinal but I have not taken that into account here. To reduce its dimension, I performed a (...
user avatar
4 votes
3 answers
164 views

Curse of dimensionality: How PCA improves my model?

After having read about the curse of dimensionality, I have been looking into a filtered version of the Superconductivity dataset. The number of dimensions is 81, so initially I thought that reducing ...
user avatar
1 vote
0 answers
18 views

How to test the significance of contribution of a variable in Factor analysis and what is the resonable rule for removing a variable from FA?

My question is about the credibility from the statistical point of view of what I have seen in some papers where the researchers (non-statistician) define some kind of staged factor analysis (FA) and ...
user avatar
0 votes
0 answers
12 views

Canonical Correlation Analysis CCA dimensionality reduction in R

I want to conduct dimensionality reduction using CCA. Let's suppose that $X$ is a matrix of $p$ predictors, $n$ samples and $y$ is a binary vector of $0$ and $1$, i.e. ...
user avatar
  • 249
8 votes
2 answers
261 views

PCA as a Cure for the Curse of Dimensionality

I would like some clarification as to how principal component analysis mitigates the Curse of Dimensionality problem. My particular interest is in curbing overfitting in my modelling, or more ...
user avatar
0 votes
0 answers
8 views

Is it possible to "warp" samples of different dimensions for sparse PCA?

It's my understanding that for $n$ samples each with $k$ dimensions I can utilize dimensionality reduction methods (ie. PCA, or in my case, sparse PCA) in a straightforward way. But what if my samples ...
user avatar
  • 151
0 votes
0 answers
18 views

Factor Analysis or PCA on part of or grouped variables

I am working on industrial time series data. (Such as sensor and controller signals, etc.) I have 150 features. Some of my features/indepentdent variables are highly correlated with each other. (75-80-...
user avatar
  • 21
1 vote
2 answers
71 views

What is the maximum number of dimensions in MDS?

If I have an arbitrary Euclidean distance matrix $D=(d_{ij}:i=1,\ldots,I; j=1,\ldots,I)$ and I want to reconstruct its elements (pairwise Euclidean distances) via classical Euclidean MDS. That is find ...
user avatar
  • 402
0 votes
0 answers
21 views

On assumptions and calculation of O'Brien's "Global Statistical Test" for Multiple Endpoints Analysis

As I understand, O'Brien's Global Statistical Test (GST) (https://pubmed.ncbi.nlm.nih.gov/6534410/) for Multiple Endpoints Analysis is a method for mapping a multivariate problem on a univariate scale....
user avatar
  • 63
0 votes
1 answer
32 views

PCA and LDA segregate the classes perfectly in 3 dimensions

I'm applying PCA and LDA on my gene expression microarray data to reduce dimensionality. It has ~20,500 features. I tried to visualize the first 3 components from both the methods in a 3d scatter plot ...
user avatar
2 votes
0 answers
69 views

Why do I get different results for the Isolation Forest after using PCA?

I am working on an unsupervised anomaly detection project and used the Isolation Forest and AutoEncoders (a normal one and a VAE) to detect anomalies. The AutoEncoders' prediction are identical but ...
user avatar
1 vote
0 answers
18 views

Are multitable analyses like Multifactor Analysis (MFA), STATIS, and Partial Triadic Analysis (PTA) JUST exploratory methods? Can I use PC Scores?

If I perform PCA on a simple table, I can take the resulting principal component scores as variables and then perform regression to predict an outcome from my original data. I would do this for ...
user avatar
0 votes
1 answer
33 views

Variance used in t-SNE

I was going through the t-SNE, but I am bit confused. While the original paper of t-SNE is based on the SNE and SNE uses $\sigma_i^2$ (note the subscript $i$) while calculating the similarity of point ...
user avatar
4 votes
4 answers
1k views

How can I reduce a very large sample size for statistical significance (sampling methods)? [duplicate]

In my biological study, I have around 14000 independent samples, and I study the evolution of a response variable over time. I have three groups to study. Thus, I have two factors: factor "Group&...
user avatar
  • 457
1 vote
0 answers
104 views

Distance metric that is robust to collinearity

I'm trying to find a distance metric that takes into account the correlation between vectors. That is, suppose we have matrix $M$ of dimensions $n \times k$, and we take the pairwise distance between ...
user avatar
0 votes
0 answers
22 views

Using dimensionality reduction to control for more variables in a multiple linear regression without breaking the 15:1 rule - possible?

I'm interested in whether there is a relationship between my particular variable of interest (X) and my dependent variable (Y). I'm planning to explore this using multiple linear regression. My focus ...
user avatar
  • 11
1 vote
1 answer
38 views

Is there an incremental dimensionality reduction algorithm that can handle batch size less than number of components to be reduced?

I have a large dataset of patient data by hour. For example, given the shape as (hours, features), patient 1 data shape could be ...
user avatar
  • 123
3 votes
0 answers
47 views

How many features/dimensions/variables can K-Nearest Neighbors handle with a finite amount of data?

KNN is an algorithm where the "Curse of Dimensionality" applies extremely literally and directly. Let's take some kind of basic, 50/50 balanced, binary classification problem. I'm wondering ...
user avatar
0 votes
0 answers
30 views

Interpreting Sparse PCA components

I am reading about Sparse PCA and I am experimenting with a dataset that consists of 9 feature variables, I am using Python Sklearn and the results I get when checking the components_ attributes is ...
user avatar
0 votes
1 answer
27 views

Practical significance of number of singular values in SVD

I am working on a binary classification problem. SVD is used for dimensionality reduction and the vector with reduced dimension is used as the feature vector. DNN is used as the classifier. There are ...
user avatar
1 vote
0 answers
23 views

Under what conditions does univariate variable prescreening fail? (random forest modeling)

Univariate statistical methods are often used to prescreen variables for possible inclusion in a model. For example, running random forests (RF) with only a single predictor at a time to prescreen ...
user avatar
  • 273
0 votes
0 answers
27 views

Does using LDA components in classification models lead to feature leakage

I have a multiclass classification problem with (too) many independent variables. And in order to reduce the number of features I did a Linear Discriminant Analysis (LDA) on my data, which created two ...
user avatar
  • 1
0 votes
0 answers
40 views

Using Partial Least Squares for reduced-dimension machine learning

I want to perform dimensionality reduction using Partial Least Squares on a complex, large-dimension data set before training various regression models on the reduced-dimension data set. I understand ...
user avatar
  • 167
0 votes
0 answers
10 views

Combining multiple datasets vs multiple models in high dimensions

This question is related to this one and this one, but I was wondering about this topic in general. Imagine a setting where multiple datasets, representing different measurements, have been gathered. ...
user avatar
0 votes
0 answers
17 views

Is there any methods to identify when PCA transformation is not optimal

I am working on unsupervized clustering problematic. I am currently reducing the dimensions of my data by applying first a PCA transformation through which I keep 80% of the variance (reducing ...
user avatar
0 votes
0 answers
44 views

Dimension reduction with categorical and quantitative data. (up to a million data entries)

I have data on T cell receptor gene usage: [1]: https://i.stack.imgur.com/z7Jj8.png Essentially this is categorical data (first 4 columns) with a quantitative frequency column. I have tried using PCA ...
user avatar
1 vote
0 answers
65 views

How is UMAP a valid dimensionality reduction technique when it uses KNN, which suffers from the curse of dimensionality?

I have not a found a satisfactory, or really any answer, to the following problem that I cannot resolve myself. UMAP is touted as an excellent dimensionality reduction technique by constructing a high-...
user avatar
  • 104
0 votes
0 answers
19 views

Using UMAP on the space of principal components, a valid proposition?

I have a genomics dataset with roughly 16,000 features. Currently, I'm in the process of clustering for cell subtype identification, which I'll then build a classifier on. However, I've run into two ...
user avatar
  • 104
0 votes
0 answers
17 views

Finding independent variables from tabulated dataset

Given a tabulated dataset for $n$ variables, how can I find the smallest subset that consists of $m$ independent variables ($m \leq n$), so that the complete dataset can be constructed by knowing only ...
user avatar
  • 101
0 votes
0 answers
24 views

How to choose the best ICA function

I am using the fastICA library from scikit-learn on a project and noticed that there was a fun parameter, referencing the ...
user avatar
0 votes
1 answer
46 views

Transforming input features for t-SNE and UMAP

Previously I was using PCA as my dimension reduction algorithm of choice, but have recently moved to using t-SNE and UMAP. For PCA I would apply transformations to my input features to ensure they ...
user avatar
  • 101
1 vote
1 answer
175 views

Reconstruction error in PCA?

I'm using PCA for a while, but recently I read about reconstruction error which I cannot understand... For example let's consider dataset consisting 5 variables: $X_1, X_2, X_3, X_4, X_5$. ...
user avatar
  • 289
1 vote
1 answer
67 views

Find first Principal Component (and loading) using a fast iterative algorithm without covariance matrix

I have a matrix $X$ and I would like to find its first principal component and the corresponding loadings. I would like to do this without computing the covariance matrix of $X$. How can I do so? This ...
user avatar
2 votes
1 answer
89 views

Principal component analysis of RT-qPCR data

Greetings to all biostatisticians, I am analyzing a gene expression data set consisting of around 100 genes that were measured by RT-qPCR and expession values are given as 2^-delta Ct. Expression of ...
user avatar
3 votes
1 answer
58 views

Is Nonnegative matrix factorization a clustering method or a dimensionality reduction method?

In the matrix factorization we have the problem of decomposing a nonnegative matrix $X$ into two lower-rank matrices $W$ and $H$. I would like to know whether this method is considered as a dimension ...
user avatar
  • 143
0 votes
0 answers
100 views

What does it mean if we use eigenvetor matrix of centered matrix $\mathbf{X}$ to reduce the dimension of $\mathbf{X}$ without centering?

Recently, I was trying to use PCA to reduce the dimension of matrix $\mathbf{X}$. Suppose a data matrix $\mathbf{X}\in \mathbb{C}^{M\times N}$, where $M$ is the number of variables and $N$ is the ...
user avatar
  • 1
0 votes
0 answers
14 views

How can I use projections (components) from multiple correspondence analysis in subsequent regression analysis, similar to PCA

I am trying to reduce the dimension of a matrix of several hundred binary/indicator/boolean variables, and then use the reduced components in subsequent regression modeling. For continuous variables, ...
user avatar
  • 25
0 votes
0 answers
36 views

What effect would clusters get when adding more variables to clustering task?

I did kmeans++ clustering for 100 clusters on user data. When I first tried clustering with two variables, I set the number of clusters to 100 and looked at the result of clustering. The number of ...
user avatar
  • 2,583
0 votes
0 answers
12 views

Can regularization be used when your features >> number of observations to reduce the feature space?

I'm wondering if a reasonable way of reducing the feature space when p >> n is to simply use l1/l2 regularization. Will this work? Or can the model simply not be fit to begin with, and so the ...
user avatar

1
2 3 4 5
21