Questions tagged [dimensionality-reduction]
Refers to techniques for reducing a large number of variables or dimensions spanned by data to a smaller number of dimensions while preserving as much information about the data as possible. Prominent methods include PCA, MDS, Isomap, etc. The two main subclasses of techniques: feature extraction and feature selection.
791
questions
0
votes
1answer
13 views
Finding the variables that explain a label through LDA and PCA
I have a dataset of about 200 continuous variables with an also continuous target variable. I want to find those predictors that explain target values that are below 50. To do that I create an ...
0
votes
0answers
5 views
Are there reasonable ways to do bidirectional dimensionality reduction?
Are there any sensible techniques for dimensionality reduction, for example from 20 to 5 dimensions, and then being able (albeit with loss of information) go back from 5 to 20?
Algorithms like t-SNE, ...
2
votes
1answer
45 views
Intuition About Principal Component Directions
I am trying to really get a deep understanding of PCA. From my understanding, a principal component is defined as
$$\mathbf{z}_k = \phi_{1,k} \mathbf{x}_1 + \ldots + \phi_{p,k} \mathbf{x}_p = \mathbf{...
0
votes
0answers
38 views
Can data ever be too high dimensional for the Lasso?
I'm trying to implement Lasso on high dimensional textual data.
Format of Data: p ~= 45,000, n~=4,000
When running the Lasso, I get a training score of 0 and the number of features selected as 0. ...
0
votes
1answer
26 views
Non-linear dimensionality reduction for detecting coordinate systems [closed]
I am trying to find a way to automatically find the appropriate coordinate system for a physical problem.
For example, in the case of a simple pendulum, polar coordinates are the most appropriate ...
1
vote
0answers
22 views
the accuracy of covariance between two high-dimensional vectors
Question
Is the covariance between high-dimensional vectors less accruate than covariance between two vectors in low-dimensional vecotrs?
I am asking this questio to check if there is a need for '...
2
votes
1answer
67 views
0
votes
0answers
12 views
Dimensionality reduction based on target value
I have a dataset including 100k high dimensional data (e.g. houses in LA) (dim=100, e.g. house parameters like area, distance to downtown, etc.). Below is the 2-component PCA representation of the ...
0
votes
0answers
15 views
Why is multidimensional scaling (PCoA) considered a linear dimensionality reduction technique?
Classic MDS (cMDS or PCoA) preserves global distances, characteristic of linear techniques. However, metric MDS seeks to minimize a cost function (stress), while non-metric MDS (nMDS) preserves only ...
0
votes
0answers
15 views
Different number of Eigen/Singular values from PCA and SVD
My understanding is that a SVD done on a raw data matrix M and a PCA done on its covariance matrix C should return the same eigen/singular values.
I have a 2736 x 356 data matrix and am using the ...
0
votes
0answers
5 views
What is subspace tracking?
I have been looking at some papers using Grassmannians in machine learning and the word 'subspace tracking' comes up often. I tried looking up what exactly is meant by this term but I can't seem to ...
2
votes
1answer
123 views
When to prefer PCA over regularization methods in regression?
When dealing with the curse of dimensionality, regularization methods seem to be clear in their intuition. All "regularization" methods can be seen as a "squeezing" of one's variables towards ...
0
votes
0answers
16 views
Non-metric multidimensional scaling, no convergence
I'm using MetaMDS from the VEGAN package to run a non-metric multidimensional scaling analysis. My stress level for 3 dimensions is in the excellent range (i.e., stress<.05), however, the model ...
0
votes
0answers
21 views
Neural network model has way more features than samples but yields good test accuracy
I am recently doing a bioinformatic machine learning project. We have over 470,000 features and only 700 training samples and 300 test samples. We used a perceptron with 1 hidden layer to train. ...
0
votes
0answers
17 views
Problem with Principal component (PCA) and Partial least squares (PLS) using R
I'm trying to reduce highly dimensional data with factor methods. I'm using Principal component analysis and Partial least squares.
From these methods I'm using the first component as a Common factor ...
0
votes
0answers
15 views
Factor analysis with categorical variables(FAMD) [duplicate]
I'm using the FAMD(factor analysis for categorical and numerical variables) function from the FactoMineR package in R. The cumulative variance of my first 4 dimensions is very small. The first 4 ...
0
votes
1answer
13 views
Feature selection on full training set, does information leak if using Filter Based Feature Selection or Linear discriminate analysis?
In order to test a potential classification set, usually some data is kept as a holdout set, and not used for inner-cross-validation or model training.
However, what happens if too many features ...
2
votes
0answers
67 views
Find correlations based on large multivariable time-based data with one output per dataset
I am not well versed in anything beyond basic statistics but have been tasked with coming up with a "grading" scale for wear on a part based on data we have collected. I am in need of help figuring ...
0
votes
0answers
17 views
Subtraction between features before performing PCA and training a classification model
In my dataset, I've created moving averages for historical indicators.
Is it pertinent to perform subtraction between those moving averages before PCA?
My stats/calculus feeling: as PCA is a ...
0
votes
0answers
23 views
EOF/ PCA: do I need to detrend my multivariate time series before finding Empirical Orthogonal Functions?
I am familiar with Principal Component Analysis, but I have recently been asked to find the Empirical Orthogonal Functions of a multivariate time series and I am not sure if what I need to do is just ...
0
votes
0answers
28 views
Is there an optimal variable to remove in a correlational dimension reduction method?
I am seeking to reduce the number of variables in my data set by finding pairs of variables which are highly correlated and therefore redundant.
Once I have determined pairs of correlated variables, ...
0
votes
1answer
48 views
When does partial least squares provide >1 component solutions?
I'm a beginner to using partial least squares analyses, so apologies if this question is a bit basic.
I've been trying out PLS models on my datasets and it usually says that a single component can ...
0
votes
1answer
24 views
using latent dirichlet allocation to reduce the number of dimensions in bag of words model?
Does anyone have experience reducing the dimensions in a traditional bag of words model?
For example, if you want to train a decision tree on a large set of reviews, the size of the vocabulary ...
0
votes
1answer
48 views
0
votes
1answer
23 views
Positioning multivariate data in a 2-dimensional space (with PCA)
I have multidimensional data. (11 columns - attributes , 150K rows - number of data). It is slightly sparse-alike data, for example, which means one datum has numeric values like (0, 0, 6.5, 0, 0, 7.5,...
2
votes
0answers
43 views
How to conduct a principal component analysis on data set with large number of zeros
I have data for percentage cover of plant species in 500 sites. There are columns for 30 different species in the data set and I would like to drastically reduce this down to a manageable number of ...
0
votes
0answers
33 views
Chi Square Test for Dimensionality Reduction
According to many resources, we should have categorical variable to be able to apply chi square test.
...
0
votes
1answer
38 views
Dimensionality Reduction - Feature Selection
For example, we have a dataset in which the samples contain 400 features. In this case, if we try to perform classification, we get very low accuracy because our learning model will become very ...
0
votes
0answers
20 views
Appropriate dimensionality reduction technique for a small, but high-dimensional sample
I am attempting to conduct some multivariate analysis on a dataset I've been given with a sample size (n) of 23 and a feature number (p) of ~800. I would like to use dimensionality reduction, but ...
1
vote
1answer
41 views
The best number of nodes in bottleneck layer in Autoencoder
I would like to perform dimensionality reduction using autoencoders (similar to PCA) and I am not sure how many components are optimal i.e. what should be the size of the bottleneck layer.
I was ...
0
votes
0answers
13 views
How can Item Response Theory be used to remove questions asked in a customer satisfaction survey?
I have results from a survey of around 30 Likert-style questions that are asked of customers on their opinion about company X.
Each of the 30 questions belongs to a certain category. For example, ...
0
votes
0answers
43 views
Principal Component Analysis - Why Use Eigenvectors of the covariance matrix? [duplicate]
In PCA we start with a dataset and we reduce its dimensions by giving it new features that are each a linear combination of the original features of the dataset, and only keeping the ones with maximum ...
1
vote
0answers
15 views
Predicting user behaviour based on transactional data - flagging “risky” behaviour
Firstly, I'm not sure if this is the right instance of StackOverflow to post on so feel free to ask me to put it elsewhere.
I am exploring the concepts of clustering and "unsupervised" learning for ...
0
votes
0answers
21 views
When to use PCA vs LDS vs nMDS for microbiota dataset?
I'm trying to understand the certain situations in which you would use the 3 above ordinance/rank tests over the other in terms of microbiota count data. Typically, I have been told to use nMDS over ...
0
votes
1answer
90 views
Correct way to calculate MSE for autoencoders with batch-training
Suppose you have a network representing an autoencoder (AE). Let's assume it has 90 inputs/outputs. I want to batch-train it with batches of size 100. I will denote my input with ...
1
vote
1answer
28 views
Finding optimal subspace for Linear Discriminant Analysis - Elements of Statistical Learning 4.3.3
Linear Discriminant Analysis (LDA) possibly operates a dimension reduction. Section 4.3.3 in Elements of Statistical Learning explicits this notion as well as a method for computing the "optimal ...
0
votes
0answers
9 views
Generating new samples from dataset to expand dataset
I want to choose one dataset and then expand the dimensions/number of samples to show how a dimension reduction method(not yet decided) reacts to changes in dimensions/number of samples. My plan was ...
3
votes
1answer
74 views
How can Factor Analysis be used to remove questions from a survey?
Suppose I have a psychological questionnaire asking 30 questions about a person's mental health (on a Likert-scale 1-7). These 30 questions fall into 7 separate, but correlated categories.
The ...
0
votes
0answers
27 views
Simplification of bivariate normal $\phi_2(x,y,\rho)$ at $y=y_F$ (i.e. fixing one of the axes)
Suppose we start off with the traditional standard bivariate normal distribution:
$$\phi_2(x,y|\rho,\mu_x=0,\mu_y=0,\sigma_x=1,\sigma_y=1)=\frac{1}{2\pi\sqrt{1-\rho^2}}\exp \left(-\frac{x^2-2\rho x y ...
0
votes
0answers
36 views
How to identify and reduce question overlap and redundancies in a survey? (remove questions asked for a more concise survey, w/o losing information)
Suppose I have a survey that contains 30 items.
The items ask about the relationship between the respondent and their family, in many different realms. For example, the strength of the connection ...
0
votes
0answers
10 views
Dimensionality reduction before clustering cosine data values causes a change of scale
In my experiment, I am doing hierarchical agglomerative clustering of texts (parameters: cosine, average). My features matrix is very sparse, so I considered PCA as dimensionality reduction technique. ...
2
votes
2answers
757 views
Intuitive explanation of how UMAP works, compared to t-SNE
I have a PhD in molecular biology. My studies recently started to involve high dimensional data analysis. I got the idea of how t-SNE works (thanks to a StatQuest video on YouTube) but can't seem to ...
0
votes
0answers
16 views
Dimension Reduction on Data with both Spatial and Non-Spatial Variables to Train a Logistic Regressor for Cross Sectional Time Series Data
I need some help on how to process and analyse a study of mine. I'm running a study on mice to look at the effect of diet on cells over a series of time. My mice are divided into two groups, one group ...
0
votes
0answers
21 views
How do you actually use PCA in MATLAB?
I'm trying to use the pca command in MATLAB for dimensionality reduction. I know that
[U, V] = pca(X) will yield the principal components in U and the scores in V,...
0
votes
0answers
44 views
What to do after feature agglomeration in Python?
I'm attempting to use FeatureAgglomeration (sklearn) package in Python to reduce the dimensionality in my dataset (which contains many collinear variables). I haven't been able to find as many ...
9
votes
1answer
176 views
t-SNE with mixed continuous and binary variables
I am currently investigating the visualisation of high-dimensional data using t-SNE. I have some data with mixed binary and continuous variables and the data appears to cluster the binary data much ...
1
vote
0answers
38 views
Are training-loss optimised embeddings useless? (help resolve a disagreement)
The aim
We are training a feed forward neural network as a regressor, with the aim of using the activations of the final layer as a type of embedding vector to represent the input examples.
The ...
0
votes
0answers
20 views
Similar plots of 2 closely related data sets
Suppose my data set contains 1000 observations and 10 variables. I consider 2 data sets: the original one and the one which contains only the first 5 variables (the number of observations is still the ...
2
votes
2answers
275 views
Why does larger perplexity tend to produce clearer clusters in t-SNE?
Why does larger perplexity tend to produce clearer clusters in t-SNE?
By reading the original paper, I learned that the perplexity in t-SNE is $2$ to the power of Shannon entropy of the conditional ...
1
vote
0answers
17 views
Help with terminology and/or method: find parameters that minimize cost function for each cluster of initial conditions?
This is a bit embarrassing but I can't even find the words to properly do a Google search.
I am doing simulations and I am trying to develop a better adaptation process. The process has seven ...
