Questions tagged [dimensionality-reduction]
Refers to techniques for reducing a large number of variables or dimensions spanned by data to a smaller number of dimensions while preserving as much information about the data as possible. Prominent methods include PCA, MDS, Isomap, etc. The two main subclasses of techniques: feature extraction and feature selection.
802
questions
-1
votes
1answer
28 views
What does PC1 mean in prcomp output?
I'm having trouble trying to understand the output of the prcomp function from package stats in ...
0
votes
0answers
14 views
Finding PCA-like directions in feature space that maximise sensitivity to a target variable
I have a fairly large space of feature variables in which I want to build a predictor for a target variable. My input dataset for training the predictor are sampled from the space using a mix of log ...
0
votes
0answers
13 views
How to combine PCA scores? [duplicate]
I was wondering if anyone knew how to combine the different PCA scores. In my dataset I have 3 PCs that explain more than 10% of the variation of shape of a bone and each one explains different ...
-1
votes
1answer
28 views
Is it meaningful to reduce dimension from 300 to 100 by PCA?
as I have only learned PCA for a short while, some problems occured when I faced practice. I am willing to accept solution or advice for the following content and great many thanks for anyone's kindly ...
1
vote
2answers
44 views
Why is there a reconstruction loss in PCA with orthonormal eigenvectors?
I've already read How to reverse PCA and reconstruct original variables from several principal components? and I understand conceptually and visually why there has to be a reconstruction loss.
...
0
votes
0answers
7 views
Joining many topic models. How to join similar topics?
I'm doing topic discovery on a large corpus of small texts.
I'm using many topic models, because not having a ground truth, each model covers a different semantic dimension.
So now I ended with more ...
0
votes
1answer
18 views
Saved PCA model produce different result
I'm using PCA to reduce my feature vector dimension. I'm saving its model and transformed output like this:
...
0
votes
0answers
9 views
reduce vector dimensionality for k nearest neighbor search search
I have 5000 512-dimensional vectors. I want to perform cosine similarity Nearest Neighbor on that vectors. I want to store them in DBSM that's why I somehow need to reduce the dimensionality of vector ...
0
votes
0answers
11 views
Treating seasonality in Partial Least Squares forecast
I've been looking for answers on this question but couldn't find concrete solutions so wanted to ask y'all.
I have been playing around trying to forecast an economic/financial-related indicator with ...
7
votes
2answers
77 views
Why is the autoencoder decoder usually the reverse architecture as the encoder?
Every autoencoder architecture I've seen has a similar architecture, mainly that the decoder is just the reverse of the encoder. If the goal of the autoencoder is low-dimensional feature learning, why ...
0
votes
1answer
56 views
Quantitatively assess if independent variables are sufficient to determine dependent variable behaviour
I have the matrix of the independent variables X with dimensions n_samples x m_features and a vector ...
0
votes
0answers
16 views
Will applying SVM in high dimensions (7) with limited training examples (41) likely lead to overfitting?
Right now I have a dataset with 41 training samples (and no testing samples either unfortunately). There are 7 features, but I've been treating the problem as a 2-D problem thus far (in other words, ...
2
votes
2answers
75 views
How to Reduce Number of Variables Before Running Random Forrest or XGBoost
I've simplified the problem I'm working on for this post, so that the focus is on the issue I'm having.
I'm trying to predict if a patient will be diagnosed with arthritis in 2019, based on the ICD-...
0
votes
1answer
49 views
Finding the variables that explain a label through LDA and PCA
I have a dataset of about 200 continuous variables with an also continuous target variable. I want to find those predictors that explain target values that are below 50. To do that I create an ...
0
votes
0answers
6 views
Are there reasonable ways to do bidirectional dimensionality reduction?
Are there any sensible techniques for dimensionality reduction, for example from 20 to 5 dimensions, and then being able (albeit with loss of information) go back from 5 to 20?
Algorithms like t-SNE, ...
2
votes
1answer
46 views
Intuition About Principal Component Directions
I am trying to really get a deep understanding of PCA. From my understanding, a principal component is defined as
$$\mathbf{z}_k = \phi_{1,k} \mathbf{x}_1 + \ldots + \phi_{p,k} \mathbf{x}_p = \mathbf{...
0
votes
0answers
45 views
Can data ever be too high dimensional for the Lasso?
I'm trying to implement Lasso on high dimensional textual data.
Format of Data: p ~= 45,000, n~=4,000
When running the Lasso, I get a training score of 0 and the number of features selected as 0. ...
0
votes
1answer
28 views
Non-linear dimensionality reduction for detecting coordinate systems [closed]
I am trying to find a way to automatically find the appropriate coordinate system for a physical problem.
For example, in the case of a simple pendulum, polar coordinates are the most appropriate ...
1
vote
0answers
25 views
the accuracy of covariance between two high-dimensional vectors
Question
Is the covariance between high-dimensional vectors less accruate than covariance between two vectors in low-dimensional vecotrs?
I am asking this questio to check if there is a need for '...
2
votes
1answer
71 views
0
votes
0answers
15 views
Dimensionality reduction based on target value
I have a dataset including 100k high dimensional data (e.g. houses in LA) (dim=100, e.g. house parameters like area, distance to downtown, etc.). Below is the 2-component PCA representation of the ...
1
vote
0answers
25 views
Is multidimensional scaling (PCoA) a linear dimensionality reduction technique?
Classic MDS (cMDS or PCoA) preserves global distances, characteristic of linear techniques. However, metric MDS seeks to minimize a cost function (stress), while non-metric MDS (nMDS) preserves only ...
0
votes
0answers
18 views
Different number of Eigen/Singular values from PCA and SVD
My understanding is that a SVD done on a raw data matrix M and a PCA done on its covariance matrix C should return the same eigen/singular values.
I have a 2736 x 356 data matrix and am using the ...
0
votes
0answers
7 views
What is subspace tracking?
I have been looking at some papers using Grassmannians in machine learning and the word 'subspace tracking' comes up often. I tried looking up what exactly is meant by this term but I can't seem to ...
2
votes
1answer
126 views
When to prefer PCA over regularization methods in regression?
When dealing with the curse of dimensionality, regularization methods seem to be clear in their intuition. All "regularization" methods can be seen as a "squeezing" of one's variables towards ...
0
votes
0answers
21 views
Non-metric multidimensional scaling, no convergence
I'm using MetaMDS from the VEGAN package to run a non-metric multidimensional scaling analysis. My stress level for 3 dimensions is in the excellent range (i.e., stress<.05), however, the model ...
0
votes
0answers
22 views
Neural network model has way more features than samples but yields good test accuracy
I am recently doing a bioinformatic machine learning project. We have over 470,000 features and only 700 training samples and 300 test samples. We used a perceptron with 1 hidden layer to train. ...
0
votes
0answers
20 views
Problem with Principal component (PCA) and Partial least squares (PLS) using R
I'm trying to reduce highly dimensional data with factor methods. I'm using Principal component analysis and Partial least squares.
From these methods I'm using the first component as a Common factor ...
0
votes
1answer
13 views
Feature selection on full training set, does information leak if using Filter Based Feature Selection or Linear discriminate analysis?
In order to test a potential classification set, usually some data is kept as a holdout set, and not used for inner-cross-validation or model training.
However, what happens if too many features ...
2
votes
0answers
67 views
Find correlations based on large multivariable time-based data with one output per dataset
I am not well versed in anything beyond basic statistics but have been tasked with coming up with a "grading" scale for wear on a part based on data we have collected. I am in need of help figuring ...
0
votes
0answers
17 views
Subtraction between features before performing PCA and training a classification model
In my dataset, I've created moving averages for historical indicators.
Is it pertinent to perform subtraction between those moving averages before PCA?
My stats/calculus feeling: as PCA is a ...
0
votes
0answers
30 views
EOF/ PCA: do I need to detrend my multivariate time series before finding Empirical Orthogonal Functions?
I am familiar with Principal Component Analysis, but I have recently been asked to find the Empirical Orthogonal Functions of a multivariate time series and I am not sure if what I need to do is just ...
0
votes
0answers
29 views
Is there an optimal variable to remove in a correlational dimension reduction method?
I am seeking to reduce the number of variables in my data set by finding pairs of variables which are highly correlated and therefore redundant.
Once I have determined pairs of correlated variables, ...
0
votes
1answer
50 views
When does partial least squares provide >1 component solutions?
I'm a beginner to using partial least squares analyses, so apologies if this question is a bit basic.
I've been trying out PLS models on my datasets and it usually says that a single component can ...
0
votes
1answer
24 views
using latent dirichlet allocation to reduce the number of dimensions in bag of words model?
Does anyone have experience reducing the dimensions in a traditional bag of words model?
For example, if you want to train a decision tree on a large set of reviews, the size of the vocabulary ...
0
votes
1answer
55 views
0
votes
1answer
23 views
Positioning multivariate data in a 2-dimensional space (with PCA)
I have multidimensional data. (11 columns - attributes , 150K rows - number of data). It is slightly sparse-alike data, for example, which means one datum has numeric values like (0, 0, 6.5, 0, 0, 7.5,...
2
votes
0answers
54 views
How to conduct a principal component analysis on data set with large number of zeros
I have data for percentage cover of plant species in 500 sites. There are columns for 30 different species in the data set and I would like to drastically reduce this down to a manageable number of ...
0
votes
0answers
51 views
Chi Square Test for Dimensionality Reduction
According to many resources, we should have categorical variable to be able to apply chi square test.
...
0
votes
1answer
39 views
Dimensionality Reduction - Feature Selection
For example, we have a dataset in which the samples contain 400 features. In this case, if we try to perform classification, we get very low accuracy because our learning model will become very ...
0
votes
0answers
21 views
Appropriate dimensionality reduction technique for a small, but high-dimensional sample
I am attempting to conduct some multivariate analysis on a dataset I've been given with a sample size (n) of 23 and a feature number (p) of ~800. I would like to use dimensionality reduction, but ...
1
vote
1answer
59 views
The best number of nodes in bottleneck layer in Autoencoder
I would like to perform dimensionality reduction using autoencoders (similar to PCA) and I am not sure how many components are optimal i.e. what should be the size of the bottleneck layer.
I was ...
0
votes
0answers
15 views
How can Item Response Theory be used to remove questions asked in a customer satisfaction survey?
I have results from a survey of around 30 Likert-style questions that are asked of customers on their opinion about company X.
Each of the 30 questions belongs to a certain category. For example, ...
0
votes
0answers
43 views
Principal Component Analysis - Why Use Eigenvectors of the covariance matrix? [duplicate]
In PCA we start with a dataset and we reduce its dimensions by giving it new features that are each a linear combination of the original features of the dataset, and only keeping the ones with maximum ...
1
vote
0answers
16 views
Predicting user behaviour based on transactional data - flagging “risky” behaviour
Firstly, I'm not sure if this is the right instance of StackOverflow to post on so feel free to ask me to put it elsewhere.
I am exploring the concepts of clustering and "unsupervised" learning for ...
0
votes
0answers
29 views
When to use PCA vs LDS vs nMDS for microbiota dataset?
I'm trying to understand the certain situations in which you would use the 3 above ordinance/rank tests over the other in terms of microbiota count data. Typically, I have been told to use nMDS over ...
0
votes
1answer
114 views
Correct way to calculate MSE for autoencoders with batch-training
Suppose you have a network representing an autoencoder (AE). Let's assume it has 90 inputs/outputs. I want to batch-train it with batches of size 100. I will denote my input with ...
1
vote
1answer
29 views
Finding optimal subspace for Linear Discriminant Analysis - Elements of Statistical Learning 4.3.3
Linear Discriminant Analysis (LDA) possibly operates a dimension reduction. Section 4.3.3 in Elements of Statistical Learning explicits this notion as well as a method for computing the "optimal ...
0
votes
0answers
10 views
Generating new samples from dataset to expand dataset
I want to choose one dataset and then expand the dimensions/number of samples to show how a dimension reduction method(not yet decided) reacts to changes in dimensions/number of samples. My plan was ...
3
votes
1answer
81 views
How can Factor Analysis be used to remove questions from a survey?
Suppose I have a psychological questionnaire asking 30 questions about a person's mental health (on a Likert-scale 1-7). These 30 questions fall into 7 separate, but correlated categories.
The ...
