Questions tagged [dimensionality-reduction]

Refers to techniques for reducing a large number of variables or dimensions spanned by data to a smaller number of dimensions while preserving as much information about the data as possible. Prominent methods include PCA, MDS, Isomap, etc. The two main subclasses of techniques: feature extraction and feature selection.

Filter by
Sorted by
Tagged with
0
votes
0answers
8 views

Maximum noise fraction (mnf)

There is a technique in remote sensing based on PCA which is called MNF-Maximum Noise Fraction(sometimes called also Minimum Noise Fraction), which from some reason is not known among other science ...
1
vote
0answers
26 views

Why does PCA result change drastically with a slight change in the input? [closed]

I am using PCA to reduce an Nx3 array to an Nx2 array. This is mainly because the PCA transformation (Nx2 matrix) is invariant to the rotations or translations performed on the original Nx3 array. Let'...
0
votes
0answers
12 views

Assumptions for Factorial Analysis for Mixed Data

Are there assumptions that must be met to perform a Factorial Analysis for Mixed Data (FAMD)? Would Bartlett's sphericity test and KMO also be used, as in PCA? Or is it not necessary, as in MCA?
0
votes
0answers
11 views

Should knn with and without PCA deliver the same results if all PCA factors are used?

I use knn with preceding PCA for dimensionality reduction. For the analysis I would use only the most "important" PCA factors but in order to validate/plausibilize my approach, I calculate ...
1
vote
0answers
23 views

Significance of Low Variance Components of PCA for Lower-Dimensional Clustering

I'm trying to apply PCA to reduce my fairly high dimensional dataset (~140 dimensions) down to 2-3 dimensions, for the explicit purposes of building an unsupervised clustering model that can be ...
0
votes
0answers
45 views

Is PCA suitable when there are continuous variables with many zeros?

I am dealing with a database where frequencies of behaviors are recorded, thus being continuous data but with many zeros. Aiming to reduce the dimensions of the seven variables, I have carried out a ...
0
votes
0answers
7 views

Unsure on how to interpret Multiple Correspondence Analysis plot ? MCA

I came across this handout: https://www.displayr.com/interpret-correspondence-analysis-plots-probably-isnt-way-think/ . Whilst it has explained alot for me, i am still uncertain on how to interpret it ...
0
votes
0answers
10 views

What technique is best for dimensionality reduction OR clustering of mixed data?

All, I have data with mixed types: categorical e.g. colour and numerical. I am trying to reduce the number of dimensions. I thought about using FACTOR ANALYSIS of mixed data, however the python ...
0
votes
0answers
17 views

Dimensionality Reduction on Mixed Data

If you have a dataset with not only numerical data, but also categorical and boolean. How do you approach doing dimensionality reduction to this dataset. The standard algorithms only work with ...
0
votes
0answers
15 views

Implication of applying PCA without reducing number of dimensions

If I understand PCA correctly, in machine learning, while usually PCA is used to conduct dimensionality reduction, it is possible that we apply the technique without reducing the number of dimensions ...
1
vote
1answer
33 views

Create composite score from categorical and binary variables

Let's say I have a dataset with a number of variables on clinical history and behaviours in the context of COVID transmission. Ultimately i'd like to create a binary variable that is an indicator of ...
2
votes
0answers
13 views

State-of-the-art methods for out-of-sample-extension

I'm using a kernel based dimensionality reduction algorithms, and interested in extending out-of-sample data points for further analysis. I've been using the Nystrom method for this task, and some ...
0
votes
0answers
9 views

Real world application of VAE for dimensionality reduction

I was reading the documentation of the UMAP package and came across this sentence: "VAEs have been shown to work only for toy datasets and to our knowledge there was no real life useful ...
1
vote
2answers
173 views

Ways to compare feature selection methods

Context: A hyperspectral image is taken (here Indiana Pines) which needs to be reduced to a lower dimension from 200 bands for this GSA is to be used. What will be possible metrics to grade various ...
1
vote
0answers
51 views

PCA when missing data are meaningful

I'd like to use Principal Components Analysis, or some other dimension reduction technique, to help visualise relationships in a multidimensional dataset. However, the dataset has missing values. I ...
0
votes
1answer
21 views

Is there any possible value in performing principal component analysis to groups separately?

I have three different groups. I did a PCA and plotted PCA1 and PCA2 as a scatter. One of the groups looks different to the others. However, afterwards I realised that I applied PCA to each group ...
0
votes
0answers
24 views

PCA and SVD not effective at capturing variance in fewer dimensions. Any good alternatives?

I have four columns, which I have z-scaled and tried PCA and SVD on, hoping that I can obtain one dimension that explains the majority of the variance. However, with PCA there is approximately 25% ...
1
vote
1answer
14 views

Is SVM uses latent variables when input variables/features are superior than 3

I was wondering if SVM uses some kind of latent variables / latent space when inputs variables/features are superior than 3. In fact I know that SVM uses dimension - 1 (a curve in 2D, a plane in 3D, .....
0
votes
0answers
29 views

Why NNMF (non-negative matrix factorization) is a method for linear dimensionality reduction?

Some sources (for example this) say that NNMF is a method for linear dimensionality reduction. How to prove this statement? I see two different explanations of this and I want to know which of them (...
0
votes
0answers
11 views

Obtaining eigenvectors of a matrix with more features than observations [duplicate]

I have a matrix A of dimension 120 x 24000, where the number of features (n=24000) represents voxels in the brain and is greater than the number of observations (m=120), or subject brain scans. I am ...
1
vote
1answer
24 views

Bayesian Information Criterion — what is the base of the logarithm?

Apologies in advance for a very basic question! On Wikipedia, I see that the Bayesian Information Criterion evaluates a model using $$ BIC = k\ln(n)- 2\ln(L) $$ where $k$ is the number of parameters, $...
0
votes
0answers
13 views

PCA, correlation of the features of projected data [duplicate]

I have just got familiar with some methods of dimensionality reduction, and one of them was PCA. So we have data $X\in \mathbb{R}^{n\times n}$ and want to reduce its dimension to $k$. PCA just takes ...
0
votes
0answers
5 views

how can i used dPCA (demixed principal component analysis) to visualize trajectories of two decision types on an INDIVIDUAL TRIAL BASIS

I have a number of awake behaving electrophysiology datasets from animals performing a decision making task (two decisions). each dataset has 30 choice trials with that are marked by a number of ...
1
vote
0answers
40 views

Guarantee of a PCA dimension estimation

Given a sample taken from a distribution concentrated near a $d$-dimensional subspace of a Euclidean space, are there published results on the probability that a PCA (principal component analysis) ...
1
vote
0answers
13 views

What is the PCR error telling me? [closed]

I tried running PCR and the results came out like this ...
1
vote
1answer
30 views

Data dimension in machine learning

I am working on Ml project and I Have 4-d dataset. I wanted to use dimensionality reduction algoritm And suddenly a question made me stop Here is my dilemma Is there difference between dimension ...
0
votes
0answers
22 views

Conditional and joint probabilities in tSNE

I have been trying to figure out why $$p_{i|j} = \frac{exp \left(-\left\|x_i - x_j\right\|^{2} / 2\sigma_i ^{2}\right)}{\sum_{k \neq i} exp \left(-\left\|x_i - x_k\right\|^{2} / 2\sigma_i ^{2}\right)}$...
2
votes
1answer
28 views

Dimensionality Reduction

I have a very basic doubt, are the number of rows/ observations in the data reduced during feature selection techniques like filter/ wrapper methods or during dimensionality reduction using PCA/ LDA/ ...
0
votes
0answers
22 views

Random Projection vs SVD

Why is the random projection computationally more efficient than Singular Value Decomposition in Dimensionality Reduction? Also, why is SVD able to retain more information?
0
votes
0answers
26 views

What is the best approach to examine the dimensionality of related behaviours and to develop theory?

I’m preparing a study whose main goal it is to explore whether a set of related behaviours are best conceptualized as one-dimensional or two-dimensional. Traditionally, such questions have been ...
0
votes
0answers
24 views

Why PCA(Principal component analysis) can reduce the linear relationship between variables

Assuming we have centralized data,the covariance matrix of the sample is X'X. This is Because: $$ Cov(X)=\frac{1}{n-1} \begin{pmatrix} X_1'X_1 &...&X_1'X_n \\ ...&...&...\\ X_n'X_1&...
3
votes
1answer
48 views

What dimensionality reduction methods allow a lower dimensional reconstruction of the original data besides PCA via invertible transformations?

In eigenfaces, one used the inverse transformation PCA is capable of doing to reconstruct the low dimensional face image. In tsne one may not reconstruct the original dataset to produce something akin ...
1
vote
1answer
35 views

PCA for Feature Selection and Calculating the Total Contributions of Each Features

First of all, I know that using PCA for feature selection is not a true approach however, I have found some articles which uses PCA for feature selection and I want to imitate them. I am having some ...
1
vote
1answer
46 views

How do I appropriately examine the dimensionality of binary data?

I have 72 binary variables and, at a theoretical level, I am trying to identify groups of variables that seem to vary together. In practice, I am struggling with how to analyze this data properly. I ...
1
vote
0answers
25 views

Dimensionality reduction in repeated measures

I'm studying data on a project that measures 12 variables each month to a group of people, with the outcome variable being a continuous scale score. As there are several variables involved, between ...
3
votes
1answer
172 views

Minimize the sum of squared perpendicular distances while computing PCA

Problem (PCA): Assume that p = 2 and the the predictors are centered. Show that the sum of squared perpendicular distances from ($x_{i1}, x_{i2})$, i = 1, 2, . . . , n to the line $a_{2}x_{1}−a_{1}x_{...
1
vote
1answer
34 views

Work around restrictions of Scikit-Learns PCA implementation [duplicate]

Scikit-Learns implementation of Principal Component Analysis has some restrictions, that are based on the svd_solver (link to docs). This means, that if i have a ...
0
votes
0answers
15 views

AutoEncoder and Unsupervised Clustering

I am working on a dataset of ~300 samples with ~5000 data-points each - ranged between 0 and 100. I am interested in: Group samples for similarity; Find the differences between groups; Would make ...
1
vote
0answers
68 views

Interpret t-SNE plot groups

I am trying to interpret some t-SNE results (using the Rtsne package in R). I used t-SNE on class probabilities from a multiclass model with 6 levels to try and visualize the relationships between the ...
1
vote
2answers
57 views

Determinant =1 constraint in PCA reconstruction Error

Let $q\leq p$. As in Tibshirani's statistical learning book, one can describe the PCA problem as optimizing the $q$-dimensional reconstruction error, given on a dataset $\{x_n\}_{n=1}^N$ in $\mathbb{...
0
votes
0answers
28 views

What can I learn about the dimensions with highest variance of a matrix $M\approx L^TR$ from looking at $L$?

I have a high-dimensional, symmetric data matrix $M\in\mathbb{R}^{d\times d}$ , which is factorized by two matrices $L, R\in \mathbb{R}^{n\times d}$ : $L^TR\approx M$, where $n$ is much smaller than $...
3
votes
1answer
75 views

What does it tell you when PCA cannot reduce the dimensionality of your dataset

I'm new to PCA and I'm trying to apply it to a dataset I have with 15 different features. I normalized my dataset before applying PCA and used the PCA method in the decomposition function from sklearn....
0
votes
2answers
53 views

Performing regression on a dataset with lots of categories

I am trying to work on a price prediction model, the attributes have lots of categories and all these categories are coded as integers. I am assuming if I build a regression model on this, the model ...
1
vote
0answers
29 views

Finding a Projection Plane in Dimensionality Reduction (e.g., Multidimensional Scaling)

I have a set of data points in high-dimensional space that I wish to map onto a lower dimension (3D or 2D). Question : How do I obtain the Projection (Hyper)Plane (e.g., its normal vector or its set ...
3
votes
1answer
444 views

PCA and variable contributions to first n dimensions

I am looking at this tutorial: Factoextra R Package: Easy Multivariate Data Analyses and Elegant Visualization Especially the contributions of the variables to the first 2 dimensions: ...
0
votes
0answers
71 views

R feature selection with LASSO in a small dataset

I have a small data set (37 observations x 23 features) and want to perform feature selection with LASSO regression in order to reduce its reduce dimensionality. To achieve this, I designed the below ...
1
vote
2answers
26 views

PCA with binary and numerical variable

How would I choose to handle having a bunch of binary variables and one numerical variable when doing PCA? My thinking was to standardize the numerical variable and let the binary variables be then ...
0
votes
0answers
13 views

Different formulations of within-class scatter matrix

If we have a dataset $X= {x_1,x_2,....,x_n}$ where all the datapoints are in $d-$dimensional feature space and there are $2$ classes $c_1$ and $c_2$ for which $n_1$ points from $X$ are for class $c_1$ ...
0
votes
0answers
24 views

What Dimensionality reduction should I choose?

I have dataset with 38k obsevations and 12 variables (11 binary and 1 numeric). I am interested in exploring different clustering techniques but before that which dimensionality reduction technique ...
4
votes
1answer
680 views

Does Attention Help with standard auto-encoders

I understand the use of attention mechanisms in the encoder-decoder for sequence-to-sequence problem such as a language translator. I am just trying to figure out whether it is possible to use ...

1
2 3 4 5
19