Skip to main content

Questions tagged [dimensionality-reduction]

Techniques for reducing a large number of variables or dimensions spanned by data to a smaller number of dimensions while preserving as much information about the data as possible. Prominent methods include PCA, Factor Analysis, MDS, Independent Component Analysis, Multiple Correspondence Analysis, Isomap, etc. The two main subclasses of techniques: feature extraction and feature selection.

Filter by
Sorted by
Tagged with
0 votes
1 answer
58 views

Matrix decomposition with constraints and weighted least squares

We have a matrix, $\mathbf{X}$, of probability distributions between 6 different results, so each row $\mathbf{x}_i$ sums to 1. We want to perform dimension reduction so that each row is a linear ...
jgf1123's user avatar
0 votes
0 answers
15 views

Is the behavior of log-likelihood and number of parameters correct in probabilistic PCA?

I am studying the behavior of Probabilistic PCA as described by Tipping and Bishop (1999). I am using the R package "Rdimtools" to help. I am puzzled about the number of parameters in the ...
Daniel Caetano's user avatar
0 votes
1 answer
28 views

Conditional Independence: Equivalent Conditions

Let $X_1$ and $X_2$ be random variables, and $R(X_1)$ be a function of $X_1$. Here are two statements: (a) $X_1\perp\!\!\!\!\perp (X_2, Y) \mid R(X_1) $ (b) $X_1\perp\!\!\!\!\perp Y \mid \{R(X_1),X_2\}...
Hepdrey's user avatar
  • 79
0 votes
0 answers
27 views

Why apply PCA instead of just removing highly correlated variables? Specially in prediction tasks [duplicate]

First of all let's assume we have variables that are correlated or highly correlated. When we apply PCA we want to reduce dimensionality, PCA works better when we have a linear correlation between the ...
Gabriel_86400's user avatar
1 vote
0 answers
29 views

Dimensionality reduction and precomputed distance matrix

I have a question about dimensionality reduction. I want to understand how methods like MDS and t-SNE work. In particular, I'd like to understand the difference when I precompute the distance matrix ...
Clemente Gotelli's user avatar
0 votes
1 answer
28 views

Applying clustering algorithms after t-SNE in R

So I'm doing my bachelor`s work and I'm applying different clustering algorithms on certain data. Before all the clustering of course I'm using a dimensionality reduction algorithm such as t-SNE for ...
user avatar
0 votes
0 answers
44 views

Dimension reduction for multivariate functional target that varies in length

I have target features consisting of about 70 different discrete time-series and I want to use this data to learn a model. Now the data is of course very high-dimensional, as I can also use a very ...
ZenDen's user avatar
  • 1
0 votes
1 answer
34 views

post processing in PCA and making sense of an example

The example is as follows: A bunch of doctors were asked to score a list of desirable characteristics of sales representatives. The questions were like: "in-depth knowledge about his/her product&...
figs_and_nuts's user avatar
0 votes
0 answers
19 views

Why does FPCA not use scaling as PCA?

Functional principal component analysis (FPCA), according to the original paper, does not use scaling before FPCA, as in PCA. Instead, it uses a covariance matrix to compute the eigen-components. I ...
Palantir's user avatar
0 votes
0 answers
48 views

Interpreting Multidimensional Scaling (MDS) Plot: Rotation and Axis Orientation

First post here, so let me know if I'm missing things. I'm working with a multidimensional scaling (MDS) plot generated in MATLAB using cmdscale(1-(corr(Betas))). Based on prior experiments, I suspect ...
McKinney Pitts's user avatar
0 votes
0 answers
27 views

Reduced Rank Regression Variant, literature request

The RRR framework is usually stated as the minimization problem $\min_{A,B} \|Y-XAB\|_2^2$, where $A$ have orthogonal columns and $A\sim p\times k$, $B\sim k\times q$ with $k<\min(p,q)$. This ...
Eric's user avatar
  • 1
0 votes
0 answers
75 views

PCA and Gower's Distance

I have a dataset with nutrient information about different ingredients. There are a total of 70 nutrients (numeric features) and 3 categorical features for a total of around 550 ingredients. I am ...
MSingh's user avatar
  • 1
3 votes
2 answers
71 views

Am I finding redundant columns in my data using Factor Analysis

I have a pandas data frame with 50 columns and 10 rows. The columns represent events and the rows are days. If an event occurs in a day, then the corresponding cell is a "1", else, is a &...
slow_learner's user avatar
0 votes
0 answers
33 views

PCA: Treating samples rather than variables as the dimensions to be reduced (will something similar to the transpose trick work?)

I have a matrix of data with approximately 200,000 samples and 30 variables. The variables have been standardized because their original units of measurement are arbitrary/irrelevant. I am interested ...
bluemouse's user avatar
  • 322
0 votes
0 answers
34 views

Appropiate ordination methods for compositional (percent) microbial species abundances explained by continous environmental variables

I aim to investigate the impact of various continuous environmental factors (such as nitrogen, phosphorus, etc.) on the composition of microbial communities within soil samples. These environmental ...
Cordex's user avatar
  • 77
2 votes
0 answers
16 views

Performing PCA with results of multiple other PCAs? Is this legit, how do I interpret it?

I'm a biology researcher, working on a manuscript that involves folks from outside of my discipline. They are using PCA to create indices of demographic variables, to categorize geographic areas. For ...
Jas Max's user avatar
  • 848
1 vote
1 answer
88 views

Nonlinear PCA vs Encoder in Autoencoders

I see that encoders have the benefit over PCA that they can transform both linear and non-linear data. However, isn't non-linear PCA designed to work with non-linear data? So, why do we still prefer ...
AliM's user avatar
  • 131
13 votes
4 answers
1k views

Why does Harrell Argue for "Ignoring Y during data reduction"?

In Regression Modeling Strategies page 79 (4.7 Data Reduction) reads: Data reduction is aimed at reducing the number of parameters to estimate in the model, without distorting statistical inference ...
purple-blade's user avatar
0 votes
0 answers
21 views

Dimensionality Reduction while Preserving Statistical and Correlational Features [duplicate]

I am working with a dataset comprising $k$ matrices, each representing different asset classes such as Stocks, Bonds, Linkers. Each matrix is of size $m \times n$, with $m$ being 10,000 simulations of ...
KingDingeling's user avatar
1 vote
2 answers
71 views

How to compare two species community matrices that were measured differently

I have two species community matrices, where each row is a site and each column is a species. The rows and columns are the same for each matrix (i.e., row 1 = site 1, column 1 = species A for both ...
aeiche01's user avatar
1 vote
0 answers
36 views

Methods for Dimensionality Reduction while Preserving Statistical and Correlational Features [closed]

I am working with a dataset comprising $k$ matrices, each representing different asset classes such as Stocks, Bonds, Linkers. Each matrix is of size $m \times n$, with $m$ being 10,000 simulations of ...
KingDingeling's user avatar
0 votes
0 answers
25 views

Normalize age frequency data for PCA

I am working on a project to forecast house ownership rates. One dataset I have consists of number of people of each age from 1-99 per geographic area code. For example, 20 people aged 1, 59 people ...
burn_burn_55's user avatar
5 votes
2 answers
186 views

How to deal with correlated variables

I would like to know how to deal with correlated variables, with this kind of correlation matrix: Is there a way to combine the correlated variables such as all the AV.. variables or FF.. variables? ...
Siueg's user avatar
  • 51
1 vote
0 answers
50 views

Examples of UMAP performing poorly

Can anyone suggest some datasets or examples that would be useful for showing situations where UMAP performs poorly, say in the sense of producing spurious clusters, or missing clusters? For t-SNE, ...
Tom Solberg's user avatar
0 votes
0 answers
9 views

Generating Composite Index by PCA for a Single Country with Many Variables

I am doing a course project where I am trying to generate an index to measure the overall level of prosperity of a single country - such as the United States. What I hope is that this index could be ...
UMW23's user avatar
  • 1
3 votes
0 answers
19 views

Feature importance in expectation maximization

The context is using EM algorithm for a mixture model - more precisely Dirichlet Multinomial Mixture, as discussed in Dirichlet Multinomial Mixtures: Generative Models for Microbial Metagenomics. One ...
Roger V.'s user avatar
  • 4,286
0 votes
1 answer
92 views

How to do dimension reduction from a variational autoencoder

I am thinking about a variational autoencoder. As far as I understand it, in the encoding section you compress to a px1 tensor and then you create a $\mu$ and $\...
user1357015's user avatar
  • 1,744
0 votes
0 answers
42 views

Efficient way to encode a set of large covariance matrices

I have a computational model that involves having a set of $K$ covariance matrices, $\{\Sigma_1, ..., \Sigma_K\}$ with each $\Sigma_i \in R^{n \times n}$. Storing all these full covariance matrices is ...
dherrera's user avatar
  • 1,268
1 vote
0 answers
133 views

Applying PCA to Time-Series Emotional Data: Validity and Interpretation Concerns

I'm currently exploring the application of Principal Component Analysis (PCA) to time-series data representing various "facial emotional expression" states (e.g., anger, happiness, sadness, ...
AltunE's user avatar
  • 111
0 votes
0 answers
18 views

Searching for a proper way to reduce the dimensionality of activations from a CNN

I am conducting an analysis to compare the similarities between different images across early and late layers in a CNN. The model I am working with is the pretrained DenseNet121 that comes with ...
LI Bing's user avatar
  • 33
0 votes
0 answers
13 views

Is it possible to have better results by PCA PCs in compare to Laplacian eigenmap

Suppouse I have a data set of the form $p = 200$ and $N = 35$. I am interesting in the multiple linear regression model train, for this reason I need somehow simplify my data. I decided to use two ...
Just do it's user avatar
1 vote
0 answers
130 views

Are population principal components scale invariant?

Are population principal components scale invariant? The answer is no. I'm not sure whether my understanding regarding the first two is correct; please, correct me if I'm wrong. Also, I don't ...
reyna's user avatar
  • 365
0 votes
0 answers
76 views

Is my understanding of dimension reduction in Linear Discriminant Analysis correct?

Here is my understanding of how dimension reduction in LDA works: We have $n$ samples each with $p$ features assigned to $k$ classes. We use the sample mean $\mu_j$ of each class and the pooled ...
Steven Gubkin's user avatar
1 vote
1 answer
331 views

Can you combine two principal components into one variable when carrying out a principal component analysis?

I am getting into and trying to learn how to use principal component analyses (PCA), and got stuck on a few things that I thought someone here might be able to help me with. What I am trying to do: I ...
Mya's user avatar
  • 11
1 vote
1 answer
116 views

Are imaginary eigenvalues a fatal flaw when doing a factor analysis?

I am running an exploratory factor analysis (EFA) in R, extracting three factors (determined via parallel analysis). ...
user399309's user avatar
3 votes
1 answer
43 views

Reducing the number of observables in Factor Analysis

I have $k$ observables (e.g. questions in a questionnaire) and $n$ observational units (e.g. respondents of the questionnaire). Let's call the observation matrix $X$. Let's assume I have performed a ...
Aleksejs Fomins's user avatar
1 vote
1 answer
32 views

How can I rigorously quantify the increase performance due to additional parameters?

I am trying to evaluate a novel dimensionality reduction technique. Specifically, we start with a data set with around 1,000 features/covariates per observation. My technique maps this down to 12. ...
golfWolf's user avatar
0 votes
0 answers
14 views

Generalized Low-Rank Models in a Regression Format

I've been using principle component regression (PCR) to model data, and was interested in using generalized low rank models (GLRM) in its place. Using PCR, I am able to easily get the coefficients for ...
neotanic_wizard's user avatar
0 votes
0 answers
37 views

Domain mapping when performing dimensionality reduction

Consider the problem of performing dimensionality reduction from a vector space $V_{1}$ of dimension M to another one, $V_{2}$, of dimension N, where N<M. Is there a technique or theory which ...
APA's user avatar
  • 1
0 votes
0 answers
58 views

Comparing scree plots or explained variance of two groups with different number of features after PCA

I want to define the dimensionality of a group as the number of PC features that can explain 80% of the variance in the group dataset. This intuition seems to work for a single group, however, if I ...
Jules's user avatar
  • 1
0 votes
0 answers
24 views

Dimensional Reduction on multi dependent variable for propensity score matching

I have an analysis project and I am not quite sure about my analysis plan. The data context are about the treatment effect with confounding variables is well-established. But it have many dependent ...
Warangkana K.'s user avatar
0 votes
0 answers
32 views

Is there a difference between Principal Component and weighted mean using PC loadings? How to get Principal Component on scale of original variables?

I was interested in doing a Principal Component analysis but returning a Principal Component on the scale of the original variables. Principal component analysis in R defaults to scaling and centering,...
JElder's user avatar
  • 919
0 votes
0 answers
72 views

interpreting t-sne scatter plot

I attempted to perform t-sne on some variables: Probability (%) of dying between age 30 and exact age 70 from illness (all)', 'Suicides per 100000 (all)', 'social_support', 'birth_health', 'freedom', '...
thingy's user avatar
  • 1
0 votes
0 answers
34 views

Linear Discriminant Analysis (LDA) with continuous value?

I wonder if LDA can be supervised with continuous value. If not, what are the closest variants for such applications?
Johnny Tam's user avatar
0 votes
0 answers
14 views

Why does Kernel PCA works with validation data?

Assume that you have a matrix $X$ and you want to do Principal Component Analysis on that data. But the data contains nonlinearities, so you decided to use Kernel Principal Component Analysis instead. ...
euraad's user avatar
  • 415
1 vote
0 answers
59 views

Nonlinear Sparse PCA

Given data $x_1, \dots, x_n \in\mathbb{R}^d$, I am looking for a nonlinear dimensionality reduction technique $f: \mathbb{R}^d \rightarrow \mathbb{R}^q$ that only uses a limited number of dimensions ...
Claudio Moneo's user avatar
1 vote
0 answers
55 views

Is this a dimensionality reduction problem?

I have an interesting (hopefully) data science problem and I want to post it here for discussion. Let's say we have a data set having ONE numerical column and FIVE categorical columns. For categorical ...
Sophia's user avatar
  • 121
0 votes
0 answers
12 views

Does the ID class vector change in Kernel Linear Discriminant Analysis?

Assume that I have a matrix $X$ that has the size $m * n$ and a class ID vector with the length $n$. If I want to apply Kernel Linear Discriminant Analysis (KLDA) onto the matrix $X$ and vector $y$, ...
euraad's user avatar
  • 415
2 votes
3 answers
525 views

Can you only apply PCA feature reduction only when there are linear relationship between the variables?

My coworker and I were discussing this. He insists that PCA only makes sense just when you know at least one variable is linear combination of the rest but I think it can be applied whenever there are ...
ADayWithoutRain's user avatar
0 votes
0 answers
12 views

How can a linear autoencoder with $h=1$ hidden unit reconstruct any rank 1 matrix?

I've had this as a homework problem as a true or false type of question and I'm trying to wrap my head around why this is true. Is the reason simply represent each datapoint as a scaled version of a ...
Oliver's user avatar
  • 290

1
2 3 4 5
23