Questions tagged [dimensionality-reduction]
Techniques for reducing a large number of variables or dimensions spanned by data to a smaller number of dimensions while preserving as much information about the data as possible. Prominent methods include PCA, Factor Analysis, MDS, Independent Component Analysis, Multiple Correspondence Analysis, Isomap, etc. The two main subclasses of techniques: feature extraction and feature selection.
1,078
questions
1
vote
0
answers
50
views
Importance of linear discriminants to classification at a given point
Many outstanding answers here detail the fundamentals of linear discriminant analysis. These include descriptions of its use in dimensionality reduction, an explanation of classification using Bayes' ...
0
votes
0
answers
27
views
Principal components regression vs other regression models following PCA
My understanding of principal components regression (PCR) is that it is a linear regression performed on all or a subset of predictors obtained via PCA. All the resources I've read only apply linear ...
1
vote
0
answers
9
views
How to properly visualize high-dimensional embeddings along with the decision boundary in 2D?
I have a number of embeddings (300-dimensional FastText vectors for each instance of each class) that I apply a classifier to (Logistic Regression for now). I want to visualize the embeddings as well ...
- 159
4
votes
0
answers
76
views
Is there any reason why there appears to be not much modern research into self organizing maps (SOMs)?
Usually there are clear advancements in ML methodologies in research especially, where I can say X method is essentially better than Y method for most datasets.
However, I recently accidentally ...
- 430
0
votes
0
answers
42
views
Is the correlation method used within your dataset problem dependent?
Question
Say I have a dataset $D$ with $N$ features that are trying to predict a target $y$. I would like to build a model from $D$ and part of that process is removing correlated columns to reduce ...
- 349
0
votes
0
answers
10
views
Is it possible to generate a projection matrix from Quadratic Discriminant Analysis?
Is it possible to generate a projection matrix from Quadratic Discriminant Analysis, as Linear Discriminant Analysis does?
I have found a library called mataveid that can do Linear Discriminant ...
- 327
4
votes
3
answers
1k
views
A way to train a model on data with a very large number of features
I have standard data: where rows are observations, and columns are features.
...
- 217
0
votes
0
answers
14
views
How to encode sparse and variable length sequential data
I have 100k historical horse races. The data is sequential in time, so I am wishing to use online learning to train an LSTM (or sequential attention model or something similar...) such that the model ...
- 229
0
votes
0
answers
16
views
What are some dimension reduction techniques applicable to sparse covariance matrices?
Suppose that I have $n\gg 500000$ observations, and I specify
$$\mathbf{y} \sim \text{Normal}(\mathbf{X}\boldsymbol{\beta},\sigma^2_y\boldsymbol{\Sigma}_y + \tau \mathbf{K}\mathbf{K}^T),$$ where $\...
- 1,809
0
votes
0
answers
36
views
A population model for PCA
We know that in econometrics it is common to work with population models and relationships. Thus, when we are faced with the data, we appeal to the analogy technique to emulate the population ...
0
votes
0
answers
9
views
How can I reduce dimensions using Partial Least Squares (PLS)?
How can I reduce dimensions using Partial Least Squares (PLS)?
My goal is to reduce dimension in the indepentend component way.
- 327
0
votes
0
answers
12
views
How to apply dimensionality reduction to a data set with outliers?
I try to apply dimensionality reduction to a multidimensional data set (with numerical features) with significant outliers. I have managed to identify outliers with Isolation Forest but now I'm in a ...
- 725
1
vote
1
answer
55
views
factanal() argument: lower
What is the purpose of the lower element in the control argument of the factanal function?
...
- 105
1
vote
0
answers
24
views
Clustering on thousands of product feature clicks and pages viewed
I want to classify 120k customers into 5-6 clusters basis the product usage, say, hundreds of product features clicked and hundreds of product pages viewed. The data will be like a customer_id has ...
- 31
1
vote
0
answers
25
views
Is there any non linear dimensionality reduction algorithm that returns a projection matrix?
Is there any non linear dimensionality reduction algorithm that returns a projection matrix $W$, such as Kernel PCA and Kernel LDA does?
The projection matrix $W$ can be a non linear transformation ...
- 327
0
votes
0
answers
20
views
Applying non-linear dimensionality reduction on binary data
Would be beneficial to apply nonlinear dimensionality reduction on a binary dataset (around 200 binary features) ? what is difference from applying MCA (multiple correspondence analysis).
8
votes
1
answer
725
views
Is it possible to turn PCA into ICA by rotating the eigenvectors?
Assume that you have made a PCA analysis and you got your eigenvectors inside the projection matrix $W$. If you project your data $X$ with $W$, then you get the desired projected dimension.
But PCA ...
- 327
2
votes
1
answer
38
views
Locality Preserving Projection (LPP) VS Principal Component Analysis (PCA)
What's the difference between Locality Preserving Projection (LPP) and Principal Component Analysis (PCA)?
This is our data. It's a 3D plot. Here I use LPP and PCA to reduce the 3D data to 2D data. It ...
- 327
1
vote
0
answers
15
views
Does min-norm least squares solve regular least squares in some basis?
For a data matrix $X$ of dimension $n \times p$ where $p > n$ and corresponding label vector $y$ of dimension $n$, the standard least squares fit, $\hat{\beta} = (X^TX)^{-1}X^Ty$, is ...
- 506
1
vote
1
answer
26
views
So how can I project the data with the eigenvectors from LDA?
I have data that look like this.
And my goal is to reduce this 3D dimension into 2D dimension so it might looks like this. Turning the angle so the distance between all classes becomes maximum.
So ...
- 327
0
votes
0
answers
38
views
Proof that PCA is equivalent to MDS when using Euclidean distances
As I was watching a video explaining how MDS works, the narrator mentioned that PCA is equivalent to MDS when Euclidean distances are used. I got confused as to how that's the case.
My guess is that ...
- 21
0
votes
0
answers
26
views
Clustering sparse dataset with mix of continuous and categorical variables
I am trying to cluster sparse heterogeneous datasets containing demographics and diagnosis variables ( mix of categorical and numerical variables).
How should I start my clustering endeavors ? start ...
0
votes
0
answers
21
views
How to do column weighted truncated SVD?
I have an unusual case where I need to combine two vector spaces but weight one more than the other. Rather than discussing my specific use case, it's likely easier to imagine we trained two word2vec ...
- 278
5
votes
1
answer
160
views
Reduce dimensions of matrices with different shapes: Methods?
BACKGROUND: I have huge (>10000x10000px) pathology slides images with different sizes. Here is an example:
You can find this specific example here. This images are pieces of tissue samples (...
0
votes
0
answers
17
views
multiple time series dimensionality reduction
I have a time series for each of a couple hundred patients, around 10-20 samples per patient, unevenly distributed through time, with over 40000 columns per sample. The target feature is the level of ...
0
votes
1
answer
164
views
How to use slopes in PCA?
I would like to use slope values in PCA. The problem I face is that the slopes I calculate per group could be within different ranges of values. We know that it is important to normalize your data ...
- 387
0
votes
0
answers
17
views
Stress value cut-off for best fit on MDS for metric data
I am drawing a multi-dimensional scaling plot using SPSS to interpret the clustering of metric data based on genetic distances. Upon using ALSCAL, I get a stress value of 0.11 for 2 dimensions and 0....
1
vote
0
answers
24
views
Dimensionality reduction that preserves non-trivial similarity
I have a set of vectors $x_i \in\mathbb{R}^m$ and a similarity function f that quantifies how similar $x_i,x_j$ are. Unfortunately, calculating f takes a lot of time.
I want to use some neural ...
- 21
0
votes
0
answers
92
views
Interpretation of a shepard plot for NMDS
I am trying to interpret an NMDS analysis. And for that I did a shepard plot. I understand that ideally the points should follow a monotonic line, which is not really the case in my example. But I do ...
- 103
0
votes
0
answers
27
views
Recommendations on best papers / blogs / existing literature on constructing risk and vulnerability indices?
I'm interested in constructing a risk index by indexing a large number of identified risk factors into a composite measure (that ideally then has sub-dimensions that can be explored further if one so ...
0
votes
0
answers
105
views
Learning to calculate output dimension from convolution and max pooling
I have tried to derive myself and read the formulas for calculating the output after convolutional operations, and max pooling.
This is what I got from the internet:
Dimensions after Max Pooling and ...
- 29
0
votes
0
answers
19
views
Larger Latent Space Dim for Point Cloud Autoencoder
So I'm trying to follow a paper that uses a AE to learn point clouds. The thing is, the dimension of the point cloud data is 3 (x, y, z), but the dimension of the latent space from what I can tell is ...
- 101
0
votes
0
answers
62
views
Optimal predictive factors
Assume I am interested in predicting a time series variable $y_t$ using a vector of possible predictors $X_t$ of dimension $N_x$. I am interested in finding the optimal $N_z < N_x$ predictive ...
- 320
1
vote
0
answers
45
views
PCA - Feature Scaling [closed]
I have been reading that the features should be standardized before performing PCA but I couldn't relate to my understanding of the same.
PCA try to project the dataset in the direction of maximum ...
- 21
0
votes
0
answers
11
views
UseTake N out of M series such that correlation is minimal
Given M timeseries, I can calculate a correlation matrix. I can sum every row and take the top N lowest scores.
However, this solution is not optimal because the row scores are calculated on the ...
- 101
0
votes
0
answers
24
views
How to cluster and visualise vectors of which the components are class indices?
Let's say I have a dataset $\boldsymbol{\mathcal{X}}$ of $N$ samples wherein each sample $\boldsymbol{x}^{(i)}\in \mathcal{X}$, $i \in {1 \ldots N}$, is described by a set of $D$ features, such that $\...
1
vote
0
answers
76
views
Finding a latent representation of a high-cardinality one-hot encoded variable [duplicate]
I am working on a clustering project on a dataset that has some numerical variables, and one categorical variable with very high cardinality (~200 values). I was thinking if it is possible to create ...
- 11
1
vote
0
answers
34
views
Linear Distance in Latent Feature Space of an AutoEncoder
I would like to perform a cluster analysis on a mixed data set containing continuous, categorical and binary data.
As I have 93 features in total, I thought it might help to use an AutoEncoder to
...
- 11
0
votes
0
answers
21
views
Interpretation of Exploratory Factor Analysis
I am conducting research in cognition, where we are reducing a large dataset. An Exploratory Factor Analaysis with a 3-factor solution (based on a scree plot) reveals factors that appear to correspond ...
- 1
1
vote
0
answers
45
views
Lasso regularization doesn't work on high dimensions?
Please see the following in highlighted (ISLR 2nd Edition pg 265):
The last sentence (p=2000 example) is very concerning to me. I was going to use Lasso in ...
- 654
0
votes
0
answers
29
views
Find a projection that minizimizes residuals on multivariate regression
The problem I'm working on is from neuroscience. We've got multiple electrodes with weakly correlated noise all sampling the same system. What are using these to do autoregression on a latent 1D ...
- 1
0
votes
0
answers
18
views
How to draw dimension reduced high dimensional gaussians in 2D for EM algorithm visualization
I'm implementing the EM algorithm. The visualization works for 2D features. I'd like to visualize it for higher dimensional data using dimension reduction(PCA)
Here k= 3. Each group of elipses are ...
0
votes
0
answers
33
views
R: Dimensionality reduction using a Genetic Algorithm
I am trying to follow this paper. Basically, there are recording a bunch of metrics for the duration of the execution of some programs and they are trying to find out which are significant. They are ...
- 101
0
votes
0
answers
44
views
How normal is "normal enough"? for a distribution [when using Symbolic Aggregate ApproXimation (SAX)]
I am comparing 4000 time-series (each with 100 points). I would like to apply SAX to discretize these time-series into distinct events in order to make comparisons between/across the time-series
My ...
1
vote
0
answers
18
views
Is there a "proper" way to keep strictly nonnegative data nonnegative when performing PCA, despite centering?
I have a question that came up in my research and I would really appreciate some guidance from someone wise in the ways of dimensionality reduction.
I have a dataset of matrices that are strictly ...
- 11
1
vote
0
answers
14
views
The meaning of having the same number of principal components as the number of predictive features and using it in my ML model [duplicate]
I have 4 predictive features and 4800 observations. I did PCA fit like this:
pca = PCA(n_components=X_train_scaled.shape[1],whiten=True)
pca.fit(X_train_scaled)
...
- 423
4
votes
1
answer
352
views
What are the best ML models for hundreds of binary features (but a non-binary response variable)?
I have hundreds of binary features, resulting in a large binary design matrix (though note that my response variable is not binary). I've tried typical models like logistic regression, KNN, and SVMs ...
- 137
1
vote
1
answer
60
views
In $k$-means, how is it NP-hard if the dimensionality of the data is at least $2$ ($d\geq 2$)?
In $k$-means, how is it NP-hard if the dimensionality of the data is at least $2$ ($d\geq 2$)? Can someone justify or give reasons to this statement?
Any guidance would be appreciated.
0
votes
0
answers
16
views
Feature selection on models with poor performance
Imagine a $p\gg n$ regression situation in which one wishes to select important features, the goal being to identify a small pool of important predictors and possibly uncover mechanisms of action in ...
- 345
1
vote
0
answers
10
views
Metric for target value homogeneity across feature distributions?
I want to find a metric that can quantify target value homogeneity across feature distributions.
Without any background knowledge, it is hard to describe exactly what I want.
Therefore, I provide an ...
- 263