Questions tagged [dimensionality-reduction]
Refers to techniques for reducing a large number of variables or dimensions spanned by data to a smaller number of dimensions while preserving as much information about the data as possible. Prominent methods include PCA, MDS, Isomap, etc. The two main subclasses of techniques: feature extraction and feature selection.
941
questions
1
vote
1answer
23 views
Data dimension in machine learning
I am working on Ml project and
I Have 4-d dataset.
I wanted to use dimensionality reduction algoritm
And suddenly a question made me stop
Here is my dilemma
Is there difference between dimension ...
0
votes
0answers
18 views
Conditional and joint probabilities in tSNE
I have been trying to figure out why
$$p_{i|j} = \frac{exp \left(-\left\|x_i - x_j\right\|^{2} / 2\sigma_i ^{2}\right)}{\sum_{k \neq i} exp \left(-\left\|x_i - x_k\right\|^{2} / 2\sigma_i ^{2}\right)}$...
2
votes
1answer
24 views
Dimensionality Reduction
I have a very basic doubt, are the number of rows/ observations in the data reduced during feature selection techniques like filter/ wrapper methods or during dimensionality reduction using PCA/ LDA/ ...
0
votes
0answers
12 views
Random Projection vs SVD
Why is the random projection computationally more efficient than Singular Value Decomposition in Dimensionality Reduction? Also, why is SVD able to retain more information?
0
votes
0answers
26 views
What is the best approach to examine the dimensionality of related behaviours and to develop theory?
I’m preparing a study whose main goal it is to explore whether a set of related behaviours are best conceptualized as one-dimensional or two-dimensional. Traditionally, such questions have been ...
0
votes
0answers
22 views
Why PCA(Principal component analysis) can reduce the linear relationship between variables
Assuming we have centralized data,the covariance matrix of the sample is X'X.
This is Because:
$$
Cov(X)=\frac{1}{n-1}
\begin{pmatrix}
X_1'X_1 &...&X_1'X_n
\\
...&...&...\\
X_n'X_1&...
3
votes
1answer
34 views
What dimensionality reduction methods allow a lower dimensional reconstruction of the original data besides PCA via invertible transformations?
In eigenfaces, one used the inverse transformation PCA is capable of doing to reconstruct the low dimensional face image.
In tsne one may not reconstruct the original dataset to produce something akin ...
1
vote
1answer
25 views
PCA for Feature Selection and Calculating the Total Contributions of Each Features
First of all, I know that using PCA for feature selection is not a true approach however, I have found some articles which uses PCA for feature selection and I want to imitate them. I am having some ...
1
vote
1answer
32 views
How do I appropriately examine the dimensionality of binary data?
I have 72 binary variables and, at a theoretical level, I am trying to identify groups of variables that seem to vary together. In practice, I am struggling with how to analyze this data properly. I ...
1
vote
0answers
16 views
Dimensionality reduction in repeated measures
I'm studying data on a project that measures 12 variables each month to a group of people, with the outcome variable being a continuous scale score. As there are several variables involved, between ...
3
votes
1answer
98 views
Minimize the sum of squared perpendicular distances while computing PCA
Problem (PCA):
Assume that p = 2 and the the predictors are centered. Show that the sum of squared perpendicular
distances from ($x_{i1}, x_{i2})$, i = 1, 2, . . . , n to the line $a_{2}x_{1}−a_{1}x_{...
1
vote
1answer
26 views
Work around restrictions of Scikit-Learns PCA implementation [duplicate]
Scikit-Learns implementation of Principal Component Analysis has some restrictions, that are based on the svd_solver (link to docs). This means, that if i have a ...
0
votes
0answers
13 views
AutoEncoder and Unsupervised Clustering
I am working on a dataset of ~300 samples with ~5000 data-points each - ranged between 0 and 100. I am interested in:
Group samples for similarity;
Find the differences between groups;
Would make ...
1
vote
0answers
65 views
Interpret t-SNE plot groups
I am trying to interpret some t-SNE results (using the Rtsne package in R). I used t-SNE on class probabilities from a multiclass model with 6 levels to try and visualize the relationships between the ...
1
vote
2answers
53 views
Determinant =1 constraint in PCA reconstruction Error
Let $q\leq p$. As in Tibshirani's statistical learning book, one can describe the PCA problem as optimizing the $q$-dimensional reconstruction error, given on a dataset $\{x_n\}_{n=1}^N$ in $\mathbb{...
0
votes
0answers
26 views
What can I learn about the dimensions with highest variance of a matrix $M\approx L^TR$ from looking at $L$?
I have a high-dimensional, symmetric data matrix $M\in\mathbb{R}^{d\times d}$ , which is factorized by two matrices $L, R\in \mathbb{R}^{n\times d}$ : $L^TR\approx M$, where $n$ is much smaller than $...
3
votes
1answer
47 views
What does it tell you when PCA cannot reduce the dimensionality of your dataset
I'm new to PCA and I'm trying to apply it to a dataset I have with 15 different features. I normalized my dataset before applying PCA and used the PCA method in the decomposition function from sklearn....
0
votes
2answers
46 views
Performing regression on a dataset with lots of categories
I am trying to work on a price prediction model, the attributes have lots of categories and all these categories are coded as integers. I am assuming if I build a regression model on this, the model ...
1
vote
0answers
23 views
Finding a Projection Plane in Dimensionality Reduction (e.g., Multidimensional Scaling)
I have a set of data points in high-dimensional space that I wish to map onto a lower dimension (3D or 2D).
Question :
How do I obtain the Projection (Hyper)Plane (e.g., its normal vector or its set ...
2
votes
1answer
103 views
PCA and variable contributions to first n dimensions
I am looking at this tutorial:
Factoextra R Package: Easy Multivariate Data Analyses and Elegant Visualization
Especially the contributions of the variables to the first 2 dimensions:
...
0
votes
0answers
32 views
R feature selection with LASSO in a small dataset
I have a small data set (37 observations x 23 features) and want to perform feature selection with LASSO regression in order to reduce its reduce dimensionality. To achieve this, I designed the below ...
1
vote
2answers
22 views
PCA with binary and numerical variable
How would I choose to handle having a bunch of binary variables and one numerical variable when doing PCA? My thinking was to standardize the numerical variable and let the binary variables be then ...
0
votes
0answers
13 views
Different formulations of within-class scatter matrix
If we have a dataset $X= {x_1,x_2,....,x_n}$ where all the datapoints are in $d-$dimensional feature space and there are $2$ classes $c_1$ and $c_2$ for which $n_1$ points from $X$ are for class $c_1$ ...
0
votes
0answers
21 views
What Dimensionality reduction should I choose?
I have dataset with 38k obsevations and 12 variables (11 binary and 1 numeric). I am interested in exploring different clustering techniques but before that which dimensionality reduction technique ...
4
votes
1answer
340 views
Does Attention Help with standard auto-encoders
I understand the use of attention mechanisms in the encoder-decoder for sequence-to-sequence problem such as a language translator.
I am just trying to figure out whether it is possible to use ...
0
votes
0answers
7 views
What is meant by subspace clustering in MFA?
The basic idea of MFA is to perform subspace clustering by assuming the covariance structure for each component of the form, $\Sigma_i = \Lambda_i \Lambda_i^T + \Psi_i$, where $\Lambda_i \in \mathbb{R}...
0
votes
1answer
80 views
How to Compute the Reconstruction error in Principal Component Analysis at lower dimensions
I have m examples and d features where m<<d. So I managed to compute the eigen value and corresponding its eigen vector ... I want to compute the reconstruction error for various value of ...
2
votes
1answer
28 views
Overview Feature Extraction in images?
I have been searching for deep feature extraction approaches for a while now, but I did not find a single paper giving me a coarse overview on this matter.
Apart from an overview, for example I would ...
0
votes
0answers
8 views
Dimensionality Reduction, Regression, and Risk Score computation
I am a beginner so this might be a silly question.
Suppose that I have a dataset containing n observations of individuals participating a research, with each observation having k variables about each ...
0
votes
0answers
27 views
PCA for reducing number of variables
I have 200 variables and about 300 subjects. I wanted to create a subset of variables based on the 200 variables I have. Basically, I wanted to reduce the amount of variables (currently 200) to a ...
0
votes
0answers
11 views
Are all dimensionality reduction techniques part of metric learning?
I was reading some survey on metric learning, and they seem to include some unsupervised dimensionality reduction technique such as PCA and LLE as a metric learning algorithm. I was confused since PCA ...
2
votes
0answers
20 views
Singular value decomposition used for dimensionality reduction in brain signal topographic data
I am trying to replicate the localizer method described in this paper (page 4). I am stuck on a step which I don't completely understand, and I would like your input and interpretation to progress.
...
1
vote
1answer
29 views
Joint optimization - Feature extraction and a classifier
I am dealing with a classification problem and high dimensional data. I am using a feature extraction method ( PCA - Principle Component Analysis) followed by a Support Vector Machine (SVM). I just ...
0
votes
0answers
5 views
Reduce dimensions maximising variance explained by grouping factor
I have a cloud of $N_{red}$ red and $N_{blue}$ blue dots in $M$ dimensional space. I have more dimensions, then dots: $(N_{red}+N_{blue})<< M$. I want to reduce dimensions to a size of $K, (K<...
0
votes
0answers
10 views
Dimensionality reduction of latent space in collaborative filtering
I have the outputs of a recommender system using collaborative filtering with a user- / item-embedding in a k-dimensional latent space. That is, I have two matrices: one with n rows and k columns, ...
2
votes
0answers
39 views
Dimensionality reduction of a large covariance matrix
I have a large covariance matrix $\Sigma$ and I am reducing its dimensionality by using a truncated eigendecomposition. $\Sigma \approx VDV^T$. I remember somewhere that you could also decompose it as ...
0
votes
1answer
66 views
High Dimensional Swiss Roll? (For Metric Learning/Dimensionality Reduction)
So I've just started a project which includes some metric learning, and came accross this swiss roll in 3D to 2D problem. Ideally, you should 'unroll' the roll.
My question is, can this be extended ...
2
votes
2answers
41 views
Understanding the determination of principal components
The idea of PCA is to find the directions (in high dimensional space) in which the essential structures (with regard to large variance, scatter) of the data lie. The assumption is that original ...
3
votes
1answer
60 views
What is embedding? (in the context of dimensionality reduction)
In the context of dimensionality reduction one often uses word embedding, which seems to me a rather technical mathematical term, which rather stands out compared to the rest of the discussion, which ...
0
votes
0answers
17 views
What are some optimal solutions when concatenating very high dimensional and low dimensional feature vectors?
I'm working on a problem where I need to concatenate feature from a resnet model and few extracted features for a end-to-end deep learning model.
Model summary:
...
3
votes
2answers
243 views
Does PCA's reconstruction error get reduced with more PCs being used?
Say that the raw data is $N$-dimensional, where $N$ is a large positive integer. If we apply PCA to the dataset, and compute the reconstruction error (in $\ell^2$-norm) using the first $d \leq N$ ...
0
votes
0answers
50 views
Which dimensionality reduction technique works well for BERT sentence embeddings?
I'm trying to cluster hundreds of text documents so that each each cluster represents a distinct topic. Instead of using topic modeling (which I know I could do too), I want to follow a two-step ...
0
votes
0answers
29 views
Computing Pairwise Distances Through PCA or SVD
What should I do to reduce an mxn (m=17, n=650,000) matrix, where m are samples and n are features of these samples, into a matrix of pairwise distances (which I will then use to generate a dendrogram)...
2
votes
1answer
44 views
Can you do confirmatory factor analysis on items with different response scale
Can one do a confirmatory factor analysis with items on different response scales.
let's say 50 items have a likert response scale from 1 to 4, and the second group of items have a likert response ...
0
votes
0answers
23 views
t-SNE can destroy separability, but can it create it?
The t-SNE algorithm is helpful for visualizing a low-dimension representation of data in dimensions too high for us to imagine. In compressing the data from $\mathbb{R}^{100}$ to $\mathbb{R}^{2}$, ...
1
vote
1answer
36 views
scaling, PCA, and clustering with non-gaussian data?
I’m feeling a bit unsure about what I’m working on as I have never dealt with this sort of data before, so I could use some feedback.
I have a large dataset that I have clustered. My approach was (1) ...
0
votes
0answers
20 views
Autoencoder Incorrect Output/Predict - Model Built and Trained. *Please Assist*
Background
Let me preface, I am new to python and machine learning.
I have been tasked with creating an autoencoder to reduce dimensionality on a made-up dataset (proof of concept).
I am working in ...
0
votes
0answers
22 views
Is there an algorithm for placing 2-dimensional embeddings into a grid so they can be displayed?
I’m using PCA to reduce images down to 2d embeddings and I’d like to display the images in a grid.
The Pudding did something like this with book covers, using tsne and a library called RasterFairy, by ...
0
votes
1answer
27 views
Optimal PCA is 0?
I am following this post (https://towardsdatascience.com/how-to-tune-hyperparameters-of-tsne-7c0596a18868) to reduce of data's dimension with PCA; however, the according optimal PCA is 0! (100% sure ...
0
votes
0answers
24 views
How vector projection works behind SVD?
I was reading a blog on mathematical intuition behind SVD. Here, author pointed out three information we get after vector decomposition.
The directions of projection — the unit vectors (v₁ and v₂) ...
