Questions tagged [dimensionality-reduction]
Refers to techniques for reducing a large number of variables or dimensions spanned by data to a smaller number of dimensions while preserving as much information about the data as possible. Prominent methods include PCA, MDS, Isomap, etc. The two main subclasses of techniques: feature extraction and feature selection.
906
questions
0
votes
0answers
11 views
Factor Analysis with more variables/parameters than observations/samples
I have a dataset containing ~150 observations/rows, and significantly more variables/columns (~1000) and would like to perform factor analysis on the data. I had one question on a theoretical level ...
1
vote
0answers
17 views
Deep Autoencoder for Narrow Dataset Feature Extraction
#Background#
Hello, just to preface my question, I am relatively new to python and to machine/deep learning in general. Part of my job is now to implement an Autoencoder that reduces the ...
0
votes
1answer
24 views
How would one use Voronoi diagrams for KMeans for high dimensional data?
I am reading Aurelien Geron's Hands on Machine Learning, and in the Unsupervised learning chapter he demonstrates how to create a Voronoi diagram after performing K-Means clustering, and produces the ...
1
vote
0answers
14 views
Correlation between principal components and external variables
Given an input dataframe with n features, and s samples, where we perform a PCA such that we then represent samples in a 1 ...
0
votes
0answers
6 views
How to project a mxn matrix (m features, n samples) onto a space generated by a mxk matrix (m features, k factors)?
I got two matrices A and B, the dimension of A is m x n, where n represents the number of samples, and m represents the number of features. Then after dimensionality reduction, e.g., by NMF, I got B, ...
0
votes
0answers
11 views
How to train a Umap to choose the best set of parameters?
I'm studying UMAP in R. I'm reading a lot of possibilities to differentiate the umap. I want to evaluate the best set of parameters for my implementation. How would you suggest me to proceed?
...
1
vote
1answer
26 views
Interpreting PCA results of first two components
I don't like the looks of my PCA graph here. PCA coordinates should be uncorrelated, yet the variance between the coordinates of the second component increases as the first component decreases. What's ...
0
votes
0answers
9 views
How can I obtain a full-conditional distribution resulting from a transformation that has a dimension reduction?
Problem statement
Suppose we have a function $h(\mathbf{s})=1-[\exp(e^{\beta_0+\beta_1(||\mathbf{s}-\mathbf{x}||)^2})]^{-1}=y$, where both s and x are $1\times2$ vectors, $y$ is a scalar, $\mathbf{s}=(...
0
votes
0answers
40 views
Differentiable PCA?
Is there a differentiable method for dimensionality reduction that is either based on PCA or has the properties of:
Mathematically or algorithmically defined, e.g. not trained like an ML model or t-...
1
vote
0answers
25 views
Generating Embeddings for Tabular Data
I have a large dataset of ? x n time series Tabular (Categorical) Data. Here is an example:
Time step 1:
...
0
votes
0answers
9 views
Insufficient n > p for forecasting with an RNN
What is the best approach to handle insufficient sample size when forecasting for multiple sequences simultaneously with an RNN?
My training set has n=956 (time points) and p=262 (sequences).
I'm ...
1
vote
0answers
17 views
What are the principal components of a set of points lying on a circle? [closed]
So if there are a set of points lying on a circle (2 dimensional), what will be the principal components given that the variance of points vary equally along any 2 perpendicular directions
0
votes
0answers
14 views
Whitening on projection matrices
The projection matrix $P = I -xx^T\in \mathbf{R}^{d \times d}$ has a zero eigenvalue and eigenvalues equal to one with multiplicity $d-1$. Is it possible to apply whitening transform on $P$ taking ...
1
vote
0answers
21 views
PCA all variables vs PCA subgroups
I have about 600 variables (no response variable) and I would like to use PCA to reduce the number of dimensions (variables). After reduction, I only want to have 100 variables. One approach I can do ...
1
vote
0answers
12 views
Why does SNE include the word Stochastic?
I understand how SNE and tSNE work, but I don't get if it is just called like that because it is a probabilistic method or because there are hidden justifications that use Stochastic Processes.
0
votes
0answers
15 views
In Probabilistic PCA, Where does the arbitrary orthogonal matrix(rotation matrix) come from?
I'm working on studying Probabilistic PCA based on the paper (Tipping & Bishop, 1999), I can follow the idea that the maximum likelihood function would reach the stationary point when the the ...
1
vote
0answers
30 views
Principal Component Analysis Question - Why do the Loadings Not Change As You Change the Number of PCs?
I am trying to teach myself how to use R Shiny and as a part of this dashboard I'm constructing, I have a section where a user performs principal component analysis and can use a slider to choose the ...
0
votes
0answers
19 views
PCA dimension reduction on correlation matrix for invertability
I have a non-singular (correlation) matrix $C$ of dimension $N{\times}N$, this is a modified version of another correlation matrix, and therefore I don't think I am able to apply any calculations on ...
0
votes
0answers
12 views
Clustering & classification of customers
I have three datasets : one about general population, one about customers for a specific brand and then one with people that were part of an advertisement event and whether or that person converted to ...
0
votes
1answer
14 views
How to reconstruct the design matrix using the principal score and loading in PCA
I was reading the principal component analysis section of ISLR. The authors define the $m$th principal component score from the $m$th principal loading vector $\phi_1=(\phi_{1m},...,\phi_{pm})$ as $...
3
votes
1answer
42 views
In KNN, why does the number of training examples needed to learn a decision boundary increase (exponentially) as the number of dimensions increases?
In book I'm reading the following is said on k-nearest neighbour algorithms:
"As the number of dimensions goes up, the number of training examples you need to locate the concept's frontiers goes ...
1
vote
0answers
21 views
Using both PCA and Logistic Regression
Is it standard practice to use PCA before using Logistic Regression? Or is it considered to be non advisable?
2
votes
1answer
45 views
Is it possible to run an EFA in R that also accounts for non-independent observations (i.e., includes random effects)?
Suppose I have the following data, in which 20 participants each rated 10 items on five different dimensions. In reality I have much more of each, but I am just using this as an example.
...
1
vote
0answers
12 views
multiple tsne (and pca) transforms on one plot
I'm reviewing a paper, in which the researcher fitted different t-sne transforms and plotted them on one plot, showing clusters in 2-d after dimensionality reduction (t-sne).
Each cluster was fitted ...
6
votes
4answers
79 views
What are some non-toy applications of autoencoders?
I haven't come across a real world application of autoencoders before. Usually, for dimensionality reduction I've used PCA or random projections instead.
Most examples I've come across of using ...
0
votes
0answers
46 views
Difference between “empirical” and “theoretical” explained variance in PCA
In prcomp library in R for instance, the "empirical" and "theoretical" explained variances for PCA are given by:
...
0
votes
0answers
34 views
How does scikit-learn handle high dimensionality in its Gaussian Mixture Model implementation?
I have a dataset of 50,000 rows that I plan to fit with scikit-learn's GMM model. The dataset has 15 features, therefore I treat each row as a vector in the space $\mathbb{R}^{15}$. My question is, ...
0
votes
0answers
23 views
Interpreting SVD on non-centered matrix
I have a very large, very sparse matrix $A \in \mathbb{N}^{n \times m}$ I'd like to perform SVD on. It is non-centered. When I center it to $A'$, I can't even fit it in memory (because $A'$ is in $\...
0
votes
0answers
13 views
Variability of K SVD components [duplicate]
Let's say I have a SVD of a matrix $A = U \Sigma V^T$, $A \in \mathbb{R}^{n \times m}$, and I'm using top-k components corresponding to $\sigma_1, ..\sigma_k$, the k largest values on the diagonal of $...
2
votes
2answers
68 views
PCA vs linear Autoencoder: features independence
Principal component analysis is a technique that extract the best orthogonal subspace in which we can project our points with less information loss, maximizing the variance.
A linear auto encoder is ...
0
votes
0answers
18 views
What dimensionality reduction technique to use for high dimensional data?
I have a dataset which contains a lot of features (>>3). For computational reasons, I would like to apply a dimensionality reduction. At this point I could use different techniques:
standard PCA
...
0
votes
0answers
7 views
definition of weights in UMAP algorithm
In the UMAP algorithm on p. 13 they define the weight between a point $x_i$ and it's j-th nearest neighbour $j \leq k$ as $$w((x_i, x_{ij})) = \exp\left\{\frac{- \max(0, d(x_i, x_{ij})- \rho_i)}{\...
0
votes
1answer
43 views
Is there an eigenfaces equivalent for PCA analysis of time series, eigen-time series?
I am trying to better understand PCA as applied to time series by drawing parallels with this explanation of PCA as applied to images of faces. In particular, I would like to visualize the resulting "...
0
votes
1answer
76 views
+50
PCA with polynomial kernel vs single layer autoencoder?
What is the relationship between PCA with polynomial kernel and a single layer autoencoder ?
What if it is a deep autoencoder?
0
votes
2answers
97 views
Linear Discriminant Analysis have Small Sample Size problem (SSS) is it n<<d
It is said that LDA has a Small Sample Size problem (SSS)
This problem arises whenever the number of samples is smaller than the dimensionality of the samples.
(Source: Chen, L.F., Liao, H.Y.M., ...
2
votes
1answer
51 views
Linear Discriminant Analysis for newbie (What is the meaning of dataset is linear separable?)
What is the meaning of "LDA dataset is linear separable"?
"the classes are non-linearly separated"
"the features have nonlinear relationships"
As I know in maths for linear equation and non-linear ...
-1
votes
1answer
24 views
Can a trained Partial Least Squares (PLS) model be used for lossy compression/encoding?
Once I have carefully trained a PLS model, I know the optimal number N of components for a regressor model.
Can those components and their coefficients be used to lossy compress the original data ...
0
votes
1answer
49 views
Calculation of Covariance for both LDA lecturer notes is different
I am reading both LDA lecturer notes and follow the example, both having different covariance calculation.
In LDA lecturer note 1 slide 19,
Did not multiple by number of samples when calculating ...
1
vote
1answer
23 views
Partial data reconstruction PCA vs Autoencoders
I'm running PCA on a image data with 4 components. Obviously I could just multiply the projections with the components to approximately recover my original data set, but I can also view each ...
1
vote
1answer
32 views
Calculate the LDA in one of the lecturer notes is wrong?
One of the lecturer notes online about LDA LDA step by step
In Slide 25.
3
votes
1answer
55 views
What does the line of LDA means in plotting a 2-D plot?
I am learning LDA and following this online lecturer note LDA steps by steps
What is the meaning of the last line and how it can be plot there?
I could not get the meaning.
0
votes
1answer
18 views
Should I Include 'Year' Column in PCA
I want to do dimensionality reduction on a dataset. One of the columns present is the Year and the values are 2000, up to 2015. When doing PCA, do you treat this column as a factor or as numeric?
0
votes
2answers
40 views
What is the meaning of Standardization in LDA/FDA?
I am reading LDA from this website LDA simple steps
It said:
What is the meaning of mean? Is the mean in column or row?
In the website it said mean in columns in X, but I think suppose to be in ...
1
vote
1answer
8 views
LDA dimensional reduction newbies (How can a single data transfer/put as a 2 datapoints in a matrix?)
How can I put a dataset as a 2 vector point?
I am reading LDA simple guide where the dataset is below.
And I need to transform to a 2 vector point as below:
How can a single data transfer/put as a 2 ...
0
votes
0answers
14 views
Dimensionality reduction with features in different scales
I want to make dimensionality reduction to data that contains features in different scales (for example height in meters and weight in Kg).
Usually I use PCoA with euclidean or bray-curtis distance, ...
0
votes
0answers
15 views
Is there any possibilty to apply Kernel PCA to coupled data fields?
Thanks to a wealth of answers from the community here (with a special thank you to amoeba), kernel PCA became much clearer to me. So far I understand that the magic of kernel PCA lies in the kernel ...
0
votes
0answers
12 views
Dimension reduction and get a dataset with the number of features that larger than the minimum of the numbers of samples and features
I'm trying to perform dimension reduction with SVD on an image dataset with the shape of roughly (3000,20000) as you see the number of each sample's features is way larger than the number of samples, ...
3
votes
1answer
21 views
PCA handling dimensions of test and train data for NeuralNetworks
I want to apply PCA to a dataset $X_{train}\in R^{n\times m}$, where $X_{train} = [ x^{(1)}_{train} x^{(2)}_{train} x^{(3)}_{train} x^{(4)}_{train} ...x^{(m)}_{train} ] $ and $x^{(i)}_{train}\in R^{...
7
votes
1answer
161 views
Why does PCA often perform comparably well to nonlinear models on nonlinear problems?
The standard justification for manifold learning is that the map from the latent to observed spaces is nonlinear. For example, here is how another StackExchange user justified Isomap over PCA:
Here ...
0
votes
0answers
18 views
Regarding LDA/FDA dimensionality reduction output
Can I know that in the context of dimensionality reduction using LDA/FDA.
LDA/FDA can start with n dimensions and end with k dimensions, where k < n.
Is that ...
