Questions tagged [dimensionality-reduction]
Refers to techniques for reducing a large number of variables or dimensions spanned by data to a smaller number of dimensions while preserving as much information about the data as possible. Prominent methods include PCA, MDS, Isomap, etc. The two main subclasses of techniques: feature extraction and feature selection.
754 questions
0
votes
0answers
5 views
Why do I get an error with this data using principal axis factoring but not minimal residual factoring?
I am using n_factors() from the "psycho" package in R to figure out the number of factors for a set of data. When I use prinicipal axis factoring I get the following error:
...
0
votes
0answers
9 views
How can I recover full dimensional VAR model coefficients after fitting a VAR model to a dimensionality reduced (via PCA) dataset?
I am using PCA to reduce dimensionality prior to fitting a multivariate time-series dataset to a VAR (vector autoregressive) model. Is there any way to convert a PCA-derived VAR model to a full ...
0
votes
1answer
27 views
Factor Analysis: Single variable contributing to several latent variables
I was wondering whether factor analysis is right tool in my scenario.
That is, I have dataset $X = (X_1, X_2, X_3, X_4)$, where $X_i$ denotes a single variable. As far as I understand factor analysis, ...
0
votes
0answers
14 views
Dimensionality Reduction for Optimally Preserving KNN
Do any dimensionality reduction techniques find embeddings which optimally preserve the K-nearest neighbors of each point? If no algorithm provably does this, are there algorithms which heuristically ...
1
vote
0answers
43 views
+50
Combining subjoint distributions to create a larger joint distribution
I am trying to construct large joint distributions through smaller joint distributions and I'm not sure how to approach the literature.
I am curious if there exists a function which can take n ...
0
votes
1answer
19 views
Running t-SNE on a data set where each sample is a variably-sized 3D point cloud [closed]
I want to perform t-SNE on a collection of individual 3D point clouds, each representing a different object, in order to see how they are distributed. Each point in each point cloud has only the x,y,z ...
0
votes
0answers
13 views
Dimension Reduction for mixed variables
I am working with a dataset which consists of both categorical (14 vars) and continuous variables (5). Each categorical variable consists of a minimum of 2 categories up to 106 categories.
The aim ...
0
votes
0answers
21 views
Number of factors in Factor Analysis of Mixed Data with FactoMineR
I'm trying to perform FAMD with FactoMineR because I want to reduce the dimensionality of my data. My data has 378 dimensions and 34K rows. Around 350 of those dimensions are categorical and the rest ...
0
votes
0answers
14 views
Supervised machine learning for dimensionality reduction of control variables in logistic regression
Is it a valid approach to use the predictions of a supervised machine learning (ML) algorithm as a form of dimensionality reduction of control variables in the context of logistic regression? ...
1
vote
1answer
22 views
Approaches to reduce dimensions (feature selection/extraction) with high dimensional count data before running tree based model
My dataset has ~100k samples and 3000 dimensions. The data are counts, anywhere between 0-8 and it's pretty sparse.
Because of 'curse of high dimension', I want to shrink the number of variables ...
1
vote
1answer
53 views
How to interpret PCA coefficients to reduce dimension
I have read about similar questions. I have data which has 68 columns and about 800 samples. The 68. column is the output the rest 67 is the input variables. I want to reduce the size of my input ...
0
votes
1answer
12 views
Linear Discriminant Analysis vocabulary question
I am doing a descriptive LDA on a dataset with two (known, easily separable) classes and many features (and many more observations). I intend to use the latent variable values as a dimensionally-...
0
votes
1answer
19 views
Is the grid in a self organizing map static?
I'm trying to write my own SOM in python, and after reading material from several sources (and watching video tutorials) I think I understand all the steps.
There is however one issue that I want to ...
0
votes
0answers
11 views
Clarification on quantification of Categorical variables
I have a countries column with 49 levels. I want to quantify it. If I run CATPCA on that column would i be able to get the quantified result. Since CatPCA is like PCA or factor analysis: it extracts ...
6
votes
1answer
112 views
Why can't t-SNE capture a simple parabola structure?
As a toy example, I used t-SNE on a simple parabola to have a representation of it in one dimension.
...
0
votes
0answers
5 views
Is sketching a method for dimensionality reduction and its relation to random projection
I want to know if sketching can be categorized as a method of dimensionality reduction and more specifically feature extraction. Also, i want to understand if its related to random projection.
0
votes
0answers
16 views
Is it possible to weight items differently in a factor analysis?
Suppose I have 100 targets that have been rated by 1000 individuals. I want to perform a PCA on those 100 targets.
Now, I'm curious if I were to take some property of the targets into account, how ...
0
votes
0answers
26 views
Similarity measure before and after dimensionality reduction or clustering
I have a dataset with 500 000 samples, each sample contains 30 features.
The values of the features are in the range 0.0 to 1.0.
...
0
votes
1answer
31 views
advantage of variational autoencoder
I know that VAE is generative model. However it is also used as a dimensionality reduction method. In this case, what are advantages of VAE??
Also I saw that well-applied vae on mnist, and it was ...
1
vote
1answer
88 views
Clustering and regression with high dimensional, mixed type data
I have been looking at several similar questions and answers discussing these issues but I cannot say there is a clear answer to what I am posting here. There seems to be a general confusion with the ...
1
vote
1answer
24 views
Reference point in projection axis of SVD (singular value decomposition)
I am watching a YouTube video on SVD, and attempting to recreate some of its examples to better understand the internal machinery of the algorithm. In one of the slides, the instructor mentions that ...
1
vote
1answer
17 views
Is it necessary to constrain the size of the neighborhood in LLE to be less than the space dimensionality?
The wikipedia entry on Locally Linear Embedding (LLE) says that LLE can be broken into stages, the first of which is to learn a barycentric linear model of the data with its $k$-nearest neighbors:
$...
2
votes
0answers
25 views
If I recover VAR model coefficients from PCA-derived coefficients, do I need to ensure that the model has zero cross-correlation in the residuals?
I am investigating how to appropriately combine PCA with VAR modeling. I am using PCA to reduce the number of vars I fit to a VAR model, and am attempting to recover the non-dim. reduced coefficients ...
2
votes
1answer
26 views
Natural Language Processing: Basic Dimension Reduction with SVD of a Co-Occurence Matrix
Given sentences
I enjoy flying.
I like NLP.
I like deep learning
We can form a Co-Occurrence Matrix as follows:
Now we can apply Singular Value Decomposition to this matrix to get
$X = U \Sigma V^...
0
votes
0answers
10 views
Predicting based on regressor measured over time
Suppose I want to predict whether a patient has post-operative complications. In addition to some 'usual' regressors, such as age and weights, I also have access to variables that are measured over ...
0
votes
1answer
29 views
How to use Factors from Exploratory Factor analysis in further analysis?
I have performed an exploratory factor analysis on a large data set as a dimension reduction technique. I have come up with 20 factors that group together my predictor variables. However, I am not ...
1
vote
0answers
33 views
Principle Components Analysis - using variance as a variable?
I am following a collaborator’s methods to analyze a set of audio recordings, and I have found that she is using principal components analysis in an unexpected way. I am confused by her approach, and ...
0
votes
1answer
39 views
Does curse of dimensionality also affect principal component analysis calculations?
Based on this post, the Big-O notation for the complexity of calculating principal components analysis is $O(p^2n+p^3)$ for a dataset of size $n$ with $p$ features.
I understand that PCA is often ...
1
vote
0answers
32 views
How would PCA run on multivariate time-series data affect phase relationships across variables?
I am running PCA on a multivariate time-series dataset using observations across time (i.e. w/out time as an explicit variable) as the design matrix. Given this setup, I've found that it is difficult ...
0
votes
1answer
17 views
Is there a way to reduce high-dimensional feature space to an array of 2d tSNEs ordered along a chosen dimension?
Let's say we have 4096-d vectors (via a CNN fully-connected layer) and often we use tSNE to visualize the space, sometimes in combo with Jonker-Volgenant to assign it to a grid. When applied to image ...
1
vote
0answers
18 views
Constraints on low dimensional representations of data
Is there literature discussing introduction of constraints to loss functions in order to specify certain structures on low dimensional representations? If so, how do they compare the efficacy of the ...
6
votes
1answer
178 views
Using PCA to reduce dimensionality of training and testing data [duplicate]
I've read so many contradicting opinions that I feel like I need to ask the question myself.
Say I use PCA on a dataset with 60 variables and find that I can explain 98% of variance with 6 principal ...
0
votes
1answer
33 views
One-hot-encoding gives untractable amount of classes
I'm performing regression on the price of bycicles based on their brand, model and submodel. These features are hierarchical: one model belongs only to one brand but one brand can have many models. ...
0
votes
0answers
12 views
is random projection a linear or non-linear feature extraction method?
The dimensionality reduction has two different types: feature selection and feature selection. As far as i know, the random projection cannot be a feature selection method. Therefore, is it a linear ...
0
votes
0answers
37 views
Determining the Direction of Eigenvectors in PCA [duplicate]
I'm using R to get the principal components for several datasets.
An example result, using prcomp yields:
...
1
vote
1answer
128 views
Can t-SNE help feature selection?
I'm training a fully connected feed forward neural network for regression.
Given one training example $(x_i, y_i)$, I need to convert the raw representation $x_i$ into an invariant representation $...
7
votes
2answers
733 views
What is the problem with $p > n$?
I know that this is the solving system of linear equation problem.
But my question is why it is a problem the number of observation is lower than the number of predictors how can that thing happen?
...
0
votes
0answers
20 views
What distinguishes the different dimensions of t-Distributed Stochastic Neighbor Embedding (tSNE)
I understand conceptually how the tSNE algorithm works in one dimension. But I am confused how this works for 2 or more dimensions (let us call the number of dimensions of the lower dimensional space ...
0
votes
0answers
17 views
Feature reduction of Biological time series signals
I have a data set of biological signals (PSG signals); the dimension of the signals is high (850 features for each sample). I am looking for the best way to reduce the dimensionality of the signals. ...
0
votes
1answer
50 views
Minimum Input Dimension for Autoencoder Neural Network
Model: Assume we want to learn patterns using an autoencoder neural network. In the simplest case, such a network is "shallow" with 1 hidden layer, takes a $d$-dimensional numerical input vector $x$, ...
1
vote
0answers
39 views
What is connection between Pearson correlation coefficient and proportion of variance explained in PCA?
PCA procedure includes SVD of Covariance matrix. Based on eigenvalues we can find a proportion of variance explained by related Principal Components (eigenvectors).
...
0
votes
0answers
19 views
How to find orthogonal projection of a d-dimensional point, d>2 on Fisher's discriminant?
We know that the number of Fisher's discriminant to find for classifying data in given dimension d into output classes K, is min(K-1,d)
Using the iris dataset as an example, we know that d=4 (petal ...
2
votes
0answers
16 views
Searching intersection of elements in subsets with approximation
Task:
Having a large number of transactions that consists of distinct elements from one large set $S$ I need to find transactions in which items have intersection with more than 20% of items in ...
9
votes
4answers
411 views
Is “random projection” strictly speaking not a projection?
Current implementations of the Random Projection algorithm reduce the dimensionality of data samples by mapping them from $\mathbb R^d$ to $\mathbb R^k$ using a $d\times k$ projection matrix $R$ whose ...
-1
votes
1answer
42 views
PCA influence of duplicates
I am using sklearn IPCA decomposition and surprised that if I delete duplicates from my dataset, the result differs from the "unclean" one.
What is the reason? As I think, the variance is the same.
...
0
votes
0answers
77 views
Using UMAP or other non-linear dimension reduction techniques on response variables prior to learning?
Background
Suppose you have a training set where the response measurements are some $N$-dimensional vectors of related measurements - in my specific case, they happen to be cell viability scores for ...
1
vote
0answers
54 views
Log-transformation of Compositional Data
I am dealing with compositional data, in a high dimension.
Each sample I have behaves like:
$$ {S}^D=\left\{\mathbf{x}=[x_1,x_2,\dots,x_D]\in\mathbb{R}^D \,\left|\, x_i>0,i=1,2,\dots,D; \sum_{i=1}...
0
votes
0answers
15 views
Implementation of Compressed Regression (examples?)
I found several articles discussing compressed regression, whether in the bayesian framework or ordinary LS/GLM (1, 2, 3 and others).
The idea seems simple, the model becomes
$$ Y = \Phi X \beta + \...
1
vote
1answer
59 views
Where is t-distribution used in t-SNE?
I am trying to learn the dimensionality reduction using t-SNE technique. After some videos and explanation I understood the idea behind it.
But I am not getting where the t-distribution is used behind ...
1
vote
0answers
42 views
t-SNE for finding nearest neighbors
I had a question about dimensionality reduction for finding nearest neighbors and was hoping someone could help me out here.
Suppose I have good features for images, say penultimate layer features ...
