Questions tagged [dimensionality-reduction]
Techniques for reducing a large number of variables or dimensions spanned by data to a smaller number of dimensions while preserving as much information about the data as possible. Prominent methods include PCA, Factor Analysis, MDS, Independent Component Analysis, Multiple Correspondence Analysis, Isomap, etc. The two main subclasses of techniques: feature extraction and feature selection.
1,107
questions
0
votes
0
answers
38
views
Dimension reduction for multivariate functional target that varies in length
I have target features consisting of about 70 different discrete time-series and I want to use this data to learn a model. Now the data is of course very high-dimensional, as I can also use a very ...
0
votes
1
answer
30
views
post processing in PCA and making sense of an example
The example is as follows:
A bunch of doctors were asked to score a list of desirable characteristics of sales representatives. The questions were like: "in-depth knowledge about his/her product&...
0
votes
0
answers
18
views
Why does FPCA not use scaling as PCA?
Functional principal component analysis (FPCA), according to the original paper, does not use scaling before FPCA, as in PCA. Instead, it uses a covariance matrix to compute the eigen-components.
I ...
0
votes
0
answers
28
views
Interpreting Multidimensional Scaling (MDS) Plot: Rotation and Axis Orientation
First post here, so let me know if I'm missing things. I'm working with a multidimensional scaling (MDS) plot generated in MATLAB using cmdscale(1-(corr(Betas))). Based on prior experiments, I suspect ...
0
votes
0
answers
24
views
Reduced Rank Regression Variant, literature request
The RRR framework is usually stated as the minimization problem $\min_{A,B} \|Y-XAB\|_2^2$, where $A$ have orthogonal columns and $A\sim p\times k$, $B\sim k\times q$ with $k<\min(p,q)$.
This ...
0
votes
0
answers
62
views
PCA and Gower's Distance
I have a dataset with nutrient information about different ingredients. There are a total of 70 nutrients (numeric features) and 3 categorical features for a total of around 550 ingredients. I am ...
0
votes
2
answers
55
views
Am I finding redundant columns in my data using Factor Analysis
I have a pandas data frame with 50 columns and 10 rows. The columns represent events and the rows are days. If an event occurs in a day, then the corresponding cell is a "1", else, is a &...
0
votes
0
answers
33
views
PCA: Treating samples rather than variables as the dimensions to be reduced (will something similar to the transpose trick work?)
I have a matrix of data with approximately 200,000 samples and 30 variables. The variables have been standardized because their original units of measurement are arbitrary/irrelevant.
I am interested ...
0
votes
0
answers
13
views
Appropiate ordination methods for compositional (percent) microbial species abundances explained by continous environmental variables
I aim to investigate the impact of various continuous environmental factors (such as nitrogen, phosphorus, etc.) on the composition of microbial communities within soil samples. These environmental ...
0
votes
0
answers
19
views
Singular Value Decomposition (SVD) and Nonnegative Matrix Factorization (NMF) as dimensionality reduction techniques
I am reading about SVD and NMF and how they can be used for dimensionality reduction of a data matrix X
. What I did not understand why the data matrix is still have the same dimensions after SVD or ...
2
votes
0
answers
16
views
Performing PCA with results of multiple other PCAs? Is this legit, how do I interpret it?
I'm a biology researcher, working on a manuscript that involves folks from outside of my discipline. They are using PCA to create indices of demographic variables, to categorize geographic areas. For ...
1
vote
1
answer
63
views
Nonlinear PCA vs Encoder in Autoencoders
I see that encoders have the benefit over PCA that they can transform both linear and non-linear data. However, isn't non-linear PCA designed to work with non-linear data? So, why do we still prefer ...
13
votes
4
answers
1k
views
Why does Harrell Argue for "Ignoring Y during data reduction"?
In Regression Modeling Strategies page 79 (4.7 Data Reduction) reads:
Data reduction is aimed at reducing the number of parameters to estimate in the model, without distorting statistical inference ...
0
votes
0
answers
21
views
Dimensionality Reduction while Preserving Statistical and Correlational Features [duplicate]
I am working with a dataset comprising $k$ matrices, each representing different asset classes such as Stocks, Bonds, Linkers. Each matrix is of size $m \times n$, with $m$ being 10,000 simulations of ...
1
vote
2
answers
54
views
How to compare two species community matrices that were measured differently
I have two species community matrices, where each row is a site and each column is a species. The rows and columns are the same for each matrix (i.e., row 1 = site 1, column 1 = species A for both ...
1
vote
0
answers
36
views
Methods for Dimensionality Reduction while Preserving Statistical and Correlational Features [closed]
I am working with a dataset comprising $k$ matrices, each representing different asset classes such as Stocks, Bonds, Linkers. Each matrix is of size $m \times n$, with $m$ being 10,000 simulations of ...
0
votes
0
answers
24
views
Normalize age frequency data for PCA
I am working on a project to forecast house ownership rates. One dataset I have consists of number of people of each age from 1-99 per geographic area code. For example, 20 people aged 1, 59 people ...
5
votes
2
answers
183
views
How to deal with correlated variables
I would like to know how to deal with correlated variables, with this kind of correlation matrix:
Is there a way to combine the correlated variables such as all the AV.. variables or FF.. variables?
...
0
votes
0
answers
47
views
Examples of UMAP performing poorly
Can anyone suggest some datasets or examples that would be useful for showing situations where UMAP performs poorly, say in the sense of producing spurious clusters, or missing clusters? For t-SNE, ...
0
votes
0
answers
8
views
Generating Composite Index by PCA for a Single Country with Many Variables
I am doing a course project where I am trying to generate an index to measure the overall level of prosperity of a single country - such as the United States. What I hope is that this index could be ...
3
votes
0
answers
15
views
Feature importance in expectation maximization
The context is using EM algorithm for a mixture model - more precisely Dirichlet Multinomial Mixture, as discussed in Dirichlet Multinomial Mixtures: Generative Models for Microbial Metagenomics. One ...
0
votes
1
answer
64
views
How to do dimension reduction from a variational autoencoder
I am thinking about a variational autoencoder. As far as I understand it, in the encoding section you compress to a px1 tensor and then you create a $\mu$ and $\...
0
votes
0
answers
37
views
Efficient way to encode a set of large covariance matrices
I have a computational model that involves having a set of $K$ covariance matrices, $\{\Sigma_1, ..., \Sigma_K\}$ with each $\Sigma_i \in R^{n \times n}$. Storing all these full covariance matrices is ...
1
vote
0
answers
133
views
Applying PCA to Time-Series Emotional Data: Validity and Interpretation Concerns
I'm currently exploring the application of Principal Component Analysis (PCA) to time-series data representing various "facial emotional expression" states (e.g., anger, happiness, sadness, ...
0
votes
0
answers
18
views
Searching for a proper way to reduce the dimensionality of activations from a CNN
I am conducting an analysis to compare the similarities between different images across early and late layers in a CNN. The model I am working with is the pretrained DenseNet121 that comes with ...
0
votes
0
answers
13
views
Is it possible to have better results by PCA PCs in compare to Laplacian eigenmap
Suppouse I have a data set of the form $p = 200$ and $N = 35$. I am interesting in the multiple linear regression model train, for this reason I need somehow simplify my data. I decided to use two ...
1
vote
0
answers
115
views
Are population principal components scale invariant?
Are population principal components scale invariant?
The answer is no.
I'm not sure whether my understanding regarding the first two is correct; please, correct me if I'm wrong. Also, I don't ...
0
votes
0
answers
75
views
Is my understanding of dimension reduction in Linear Discriminant Analysis correct?
Here is my understanding of how dimension reduction in LDA works:
We have $n$ samples each with $p$ features assigned to $k$ classes.
We use the sample mean $\mu_j$ of each class and the pooled ...
1
vote
1
answer
107
views
Are imaginary eigenvalues a fatal flaw when doing a factor analysis?
I am running an exploratory factor analysis (EFA) in R, extracting three factors (determined via parallel analysis).
...
1
vote
1
answer
40
views
Reducing the number of observables in Factor Analysis
I have $k$ observables (e.g. questions in a questionnaire) and $n$ observational units (e.g. respondents of the questionnaire). Let's call the observation matrix $X$.
Let's assume I have performed a ...
1
vote
1
answer
32
views
How can I rigorously quantify the increase performance due to additional parameters?
I am trying to evaluate a novel dimensionality reduction technique. Specifically, we start with a data set with around 1,000 features/covariates per observation. My technique maps this down to 12. ...
0
votes
0
answers
13
views
Generalized Low-Rank Models in a Regression Format
I've been using principle component regression (PCR) to model data, and was interested in using generalized low rank models (GLRM) in its place. Using PCR, I am able to easily get the coefficients for ...
0
votes
0
answers
37
views
Domain mapping when performing dimensionality reduction
Consider the problem of performing dimensionality reduction from a vector space $V_{1}$ of dimension M to another one, $V_{2}$, of dimension N, where N<M.
Is there a technique or theory which ...
0
votes
0
answers
54
views
Comparing scree plots or explained variance of two groups with different number of features after PCA
I want to define the dimensionality of a group as the number of PC features that can explain 80% of the variance in the group dataset. This intuition seems to work for a single group, however, if I ...
0
votes
0
answers
23
views
Dimensional Reduction on multi dependent variable for propensity score matching
I have an analysis project and I am not quite sure about my analysis plan.
The data context are about the treatment effect with confounding variables is well-established. But it have many dependent ...
0
votes
0
answers
25
views
Is there a difference between Principal Component and weighted mean using PC loadings? How to get Principal Component on scale of original variables?
I was interested in doing a Principal Component analysis but returning a Principal Component on the scale of the original variables. Principal component analysis in R defaults to scaling and centering,...
0
votes
0
answers
60
views
interpreting t-sne scatter plot
I attempted to perform t-sne on some variables: Probability (%) of dying between age 30 and exact age 70 from illness (all)', 'Suicides per 100000 (all)', 'social_support', 'birth_health', 'freedom', '...
0
votes
0
answers
30
views
Linear Discriminant Analysis (LDA) with continuous value?
I wonder if LDA can be supervised with continuous value.
If not, what are the closest variants for such applications?
0
votes
0
answers
14
views
Why does Kernel PCA works with validation data?
Assume that you have a matrix $X$ and you want to do Principal Component Analysis on that data. But the data contains nonlinearities, so you decided to use Kernel Principal Component Analysis instead.
...
1
vote
0
answers
59
views
Nonlinear Sparse PCA
Given data $x_1, \dots, x_n \in\mathbb{R}^d$, I am looking for a nonlinear dimensionality reduction technique $f: \mathbb{R}^d \rightarrow \mathbb{R}^q$ that only uses a limited number of dimensions ...
1
vote
0
answers
55
views
Is this a dimensionality reduction problem?
I have an interesting (hopefully) data science problem and I want to post it here for discussion.
Let's say we have a data set having ONE numerical column and FIVE categorical columns. For categorical ...
0
votes
0
answers
12
views
Does the ID class vector change in Kernel Linear Discriminant Analysis?
Assume that I have a matrix $X$ that has the size $m * n$ and a class ID vector with the length $n$.
If I want to apply Kernel Linear Discriminant Analysis (KLDA) onto the matrix $X$ and vector $y$, ...
2
votes
3
answers
479
views
Can you only apply PCA feature reduction only when there are linear relationship between the variables?
My coworker and I were discussing this. He insists that PCA only makes sense just when you know at least one variable is linear combination of the rest but I think it can be applied whenever there are ...
0
votes
0
answers
12
views
How can a linear autoencoder with $h=1$ hidden unit reconstruct any rank 1 matrix?
I've had this as a homework problem as a true or false type of question and I'm trying to wrap my head around why this is true.
Is the reason simply represent each datapoint as a scaled version of a ...
0
votes
0
answers
24
views
Understanding PCA plot built on data normalized by two normalization methods
I tried two different normalization methods, generated the PCA plot above on the combined data, and colored the samples by the normalization type. Both normalization methods should give similar ...
0
votes
0
answers
28
views
Optimization of SOM grid and number of metaclusters flowSOM
I am currently attempting to optimize the dimension of the SOM grid, as well as the number of metaclusters for some flow cytometry data in R, using the well documented flowSOM package.
From what I ...
2
votes
1
answer
94
views
Why does Non-negative Matrix Factorization not give me 100% R^2 at full rank?
IMPORTANT: Please read the question carefully before labeling it a "code question" & sending it to stack overflow. The question is in fact a theoretical stats question (barring the ...
0
votes
0
answers
11
views
Linear autoencoders - Will they only preserve linear separable data?
I'm looking for to compress an image $X$ into a smaller image $x$. But not only compress, also reduce its view into a simplier view that are linear separable.
My question is: Can I use linear ...
0
votes
0
answers
35
views
What are the general steps of a semi-supervised setting?
Let's consider I have 1000 unlabeled data points. Out of these 1000 points, I manually labeled 200. Now, I am feeling lazy and don't want to label manually the remaining 800 data points.
What would be ...
0
votes
0
answers
33
views
Why do we expect the eigenvalues of the Gramian matrix found by Maximum Variance Unfolding to capture the number of degrees of freedom in the data?
Maximum Variance Unfolding (MVU) is a manifold learning method which, like other forms of dimensionality reduction, makes the assumption that whatever (high-dimensional) data we're dealing with "...