As of May 31, 2023, we have updated our Code of Conduct.

Questions tagged [dimensionality-reduction]

Techniques for reducing a large number of variables or dimensions spanned by data to a smaller number of dimensions while preserving as much information about the data as possible. Prominent methods include PCA, Factor Analysis, MDS, Independent Component Analysis, Multiple Correspondence Analysis, Isomap, etc. The two main subclasses of techniques: feature extraction and feature selection.

Filter by
Sorted by
Tagged with
1 vote
0 answers
50 views

Importance of linear discriminants to classification at a given point

Many outstanding answers here detail the fundamentals of linear discriminant analysis. These include descriptions of its use in dimensionality reduction, an explanation of classification using Bayes' ...
athankfulguest's user avatar
0 votes
0 answers
27 views

Principal components regression vs other regression models following PCA

My understanding of principal components regression (PCR) is that it is a linear regression performed on all or a subset of predictors obtained via PCA. All the resources I've read only apply linear ...
MindOfAaron's user avatar
1 vote
0 answers
9 views

How to properly visualize high-dimensional embeddings along with the decision boundary in 2D?

I have a number of embeddings (300-dimensional FastText vectors for each instance of each class) that I apply a classifier to (Logistic Regression for now). I want to visualize the embeddings as well ...
Metrician's user avatar
  • 159
4 votes
0 answers
76 views

Is there any reason why there appears to be not much modern research into self organizing maps (SOMs)?

Usually there are clear advancements in ML methodologies in research especially, where I can say X method is essentially better than Y method for most datasets. However, I recently accidentally ...
tisPrimeTime's user avatar
0 votes
0 answers
42 views

Is the correlation method used within your dataset problem dependent?

Question Say I have a dataset $D$ with $N$ features that are trying to predict a target $y$. I would like to build a model from $D$ and part of that process is removing correlated columns to reduce ...
Connor's user avatar
  • 349
0 votes
0 answers
10 views

Is it possible to generate a projection matrix from Quadratic Discriminant Analysis?

Is it possible to generate a projection matrix from Quadratic Discriminant Analysis, as Linear Discriminant Analysis does? I have found a library called mataveid that can do Linear Discriminant ...
euraad's user avatar
  • 327
4 votes
3 answers
1k views

A way to train a model on data with a very large number of features

I have standard data: where rows are observations, and columns are features. ...
mr.T's user avatar
  • 217
0 votes
0 answers
14 views

How to encode sparse and variable length sequential data

I have 100k historical horse races. The data is sequential in time, so I am wishing to use online learning to train an LSTM (or sequential attention model or something similar...) such that the model ...
Harry Stuart's user avatar
0 votes
0 answers
16 views

What are some dimension reduction techniques applicable to sparse covariance matrices?

Suppose that I have $n\gg 500000$ observations, and I specify $$\mathbf{y} \sim \text{Normal}(\mathbf{X}\boldsymbol{\beta},\sigma^2_y\boldsymbol{\Sigma}_y + \tau \mathbf{K}\mathbf{K}^T),$$ where $\...
Ron Snow's user avatar
  • 1,809
0 votes
0 answers
36 views

A population model for PCA

We know that in econometrics it is common to work with population models and relationships. Thus, when we are faced with the data, we appeal to the analogy technique to emulate the population ...
André Goulart's user avatar
0 votes
0 answers
9 views

How can I reduce dimensions using Partial Least Squares (PLS)?

How can I reduce dimensions using Partial Least Squares (PLS)? My goal is to reduce dimension in the indepentend component way.
euraad's user avatar
  • 327
0 votes
0 answers
12 views

How to apply dimensionality reduction to a data set with outliers?

I try to apply dimensionality reduction to a multidimensional data set (with numerical features) with significant outliers. I have managed to identify outliers with Isolation Forest but now I'm in a ...
Fredrik's user avatar
  • 725
1 vote
1 answer
55 views

factanal() argument: lower

What is the purpose of the lower element in the control argument of the factanal function? ...
Sam Min's user avatar
  • 105
1 vote
0 answers
24 views

Clustering on thousands of product feature clicks and pages viewed

I want to classify 120k customers into 5-6 clusters basis the product usage, say, hundreds of product features clicked and hundreds of product pages viewed. The data will be like a customer_id has ...
Nebulum's user avatar
  • 31
1 vote
0 answers
25 views

Is there any non linear dimensionality reduction algorithm that returns a projection matrix?

Is there any non linear dimensionality reduction algorithm that returns a projection matrix $W$, such as Kernel PCA and Kernel LDA does? The projection matrix $W$ can be a non linear transformation ...
euraad's user avatar
  • 327
0 votes
0 answers
20 views

Applying non-linear dimensionality reduction on binary data

Would be beneficial to apply nonlinear dimensionality reduction on a binary dataset (around 200 binary features) ? what is difference from applying MCA (multiple correspondence analysis).
Solomon123's user avatar
8 votes
1 answer
725 views

Is it possible to turn PCA into ICA by rotating the eigenvectors?

Assume that you have made a PCA analysis and you got your eigenvectors inside the projection matrix $W$. If you project your data $X$ with $W$, then you get the desired projected dimension. But PCA ...
euraad's user avatar
  • 327
2 votes
1 answer
38 views

Locality Preserving Projection (LPP) VS Principal Component Analysis (PCA)

What's the difference between Locality Preserving Projection (LPP) and Principal Component Analysis (PCA)? This is our data. It's a 3D plot. Here I use LPP and PCA to reduce the 3D data to 2D data. It ...
euraad's user avatar
  • 327
1 vote
0 answers
15 views

Does min-norm least squares solve regular least squares in some basis?

For a data matrix $X$ of dimension $n \times p$ where $p > n$ and corresponding label vector $y$ of dimension $n$, the standard least squares fit, $\hat{\beta} = (X^TX)^{-1}X^Ty$, is ...
Seraf Fej's user avatar
  • 506
1 vote
1 answer
26 views

So how can I project the data with the eigenvectors from LDA?

I have data that look like this. And my goal is to reduce this 3D dimension into 2D dimension so it might looks like this. Turning the angle so the distance between all classes becomes maximum. So ...
euraad's user avatar
  • 327
0 votes
0 answers
38 views

Proof that PCA is equivalent to MDS when using Euclidean distances

As I was watching a video explaining how MDS works, the narrator mentioned that PCA is equivalent to MDS when Euclidean distances are used. I got confused as to how that's the case. My guess is that ...
baoiba's user avatar
  • 21
0 votes
0 answers
26 views

Clustering sparse dataset with mix of continuous and categorical variables

I am trying to cluster sparse heterogeneous datasets containing demographics and diagnosis variables ( mix of categorical and numerical variables). How should I start my clustering endeavors ? start ...
Solomon123's user avatar
0 votes
0 answers
21 views

How to do column weighted truncated SVD?

I have an unusual case where I need to combine two vector spaces but weight one more than the other. Rather than discussing my specific use case, it's likely easier to imagine we trained two word2vec ...
Josh's user avatar
  • 278
5 votes
1 answer
160 views

Reduce dimensions of matrices with different shapes: Methods?

BACKGROUND: I have huge (>10000x10000px) pathology slides images with different sizes. Here is an example: You can find this specific example here. This images are pieces of tissue samples (...
Joan Gibert Fernandez's user avatar
0 votes
0 answers
17 views

multiple time series dimensionality reduction

I have a time series for each of a couple hundred patients, around 10-20 samples per patient, unevenly distributed through time, with over 40000 columns per sample. The target feature is the level of ...
Jonathan Oren's user avatar
0 votes
1 answer
164 views

How to use slopes in PCA?

I would like to use slope values in PCA. The problem I face is that the slopes I calculate per group could be within different ranges of values. We know that it is important to normalize your data ...
Quinten's user avatar
  • 387
0 votes
0 answers
17 views

Stress value cut-off for best fit on MDS for metric data

I am drawing a multi-dimensional scaling plot using SPSS to interpret the clustering of metric data based on genetic distances. Upon using ALSCAL, I get a stress value of 0.11 for 2 dimensions and 0....
Shenali Avishka Ranasinghe's user avatar
1 vote
0 answers
24 views

Dimensionality reduction that preserves non-trivial similarity

I have a set of vectors $x_i \in\mathbb{R}^m$ and a similarity function f that quantifies how similar $x_i,x_j$ are. Unfortunately, calculating f takes a lot of time. I want to use some neural ...
username_94's user avatar
0 votes
0 answers
92 views

Interpretation of a shepard plot for NMDS

I am trying to interpret an NMDS analysis. And for that I did a shepard plot. I understand that ideally the points should follow a monotonic line, which is not really the case in my example. But I do ...
C. Guff's user avatar
  • 103
0 votes
0 answers
27 views

Recommendations on best papers / blogs / existing literature on constructing risk and vulnerability indices?

I'm interested in constructing a risk index by indexing a large number of identified risk factors into a composite measure (that ideally then has sub-dimensions that can be explored further if one so ...
0 votes
0 answers
105 views

Learning to calculate output dimension from convolution and max pooling

I have tried to derive myself and read the formulas for calculating the output after convolutional operations, and max pooling. This is what I got from the internet: Dimensions after Max Pooling and ...
Mah Neh's user avatar
  • 29
0 votes
0 answers
19 views

Larger Latent Space Dim for Point Cloud Autoencoder

So I'm trying to follow a paper that uses a AE to learn point clouds. The thing is, the dimension of the point cloud data is 3 (x, y, z), but the dimension of the latent space from what I can tell is ...
okayatcp12's user avatar
0 votes
0 answers
62 views

Optimal predictive factors

Assume I am interested in predicting a time series variable $y_t$ using a vector of possible predictors $X_t$ of dimension $N_x$. I am interested in finding the optimal $N_z < N_x$ predictive ...
fes's user avatar
  • 320
1 vote
0 answers
45 views

PCA - Feature Scaling [closed]

I have been reading that the features should be standardized before performing PCA but I couldn't relate to my understanding of the same. PCA try to project the dataset in the direction of maximum ...
the_why_guy's user avatar
0 votes
0 answers
11 views

UseTake N out of M series such that correlation is minimal

Given M timeseries, I can calculate a correlation matrix. I can sum every row and take the top N lowest scores. However, this solution is not optimal because the row scores are calculated on the ...
markmarijnissen's user avatar
0 votes
0 answers
24 views

How to cluster and visualise vectors of which the components are class indices?

Let's say I have a dataset $\boldsymbol{\mathcal{X}}$ of $N$ samples wherein each sample $\boldsymbol{x}^{(i)}\in \mathcal{X}$, $i \in {1 \ldots N}$, is described by a set of $D$ features, such that $\...
stupid-question-asker's user avatar
1 vote
0 answers
76 views

Finding a latent representation of a high-cardinality one-hot encoded variable [duplicate]

I am working on a clustering project on a dataset that has some numerical variables, and one categorical variable with very high cardinality (~200 values). I was thinking if it is possible to create ...
ockham_blade's user avatar
1 vote
0 answers
34 views

Linear Distance in Latent Feature Space of an AutoEncoder

I would like to perform a cluster analysis on a mixed data set containing continuous, categorical and binary data. As I have 93 features in total, I thought it might help to use an AutoEncoder to ...
Guybrush's user avatar
0 votes
0 answers
21 views

Interpretation of Exploratory Factor Analysis

I am conducting research in cognition, where we are reducing a large dataset. An Exploratory Factor Analaysis with a 3-factor solution (based on a scree plot) reveals factors that appear to correspond ...
NicMcK's user avatar
  • 1
1 vote
0 answers
45 views

Lasso regularization doesn't work on high dimensions?

Please see the following in highlighted (ISLR 2nd Edition pg 265): The last sentence (p=2000 example) is very concerning to me. I was going to use Lasso in ...
Katsu's user avatar
  • 654
0 votes
0 answers
29 views

Find a projection that minizimizes residuals on multivariate regression

The problem I'm working on is from neuroscience. We've got multiple electrodes with weakly correlated noise all sampling the same system. What are using these to do autoregression on a latent 1D ...
Jojker's user avatar
  • 1
0 votes
0 answers
18 views

How to draw dimension reduced high dimensional gaussians in 2D for EM algorithm visualization

I'm implementing the EM algorithm. The visualization works for 2D features. I'd like to visualize it for higher dimensional data using dimension reduction(PCA) Here k= 3. Each group of elipses are ...
Houou In Kyouma's user avatar
0 votes
0 answers
33 views

R: Dimensionality reduction using a Genetic Algorithm

I am trying to follow this paper. Basically, there are recording a bunch of metrics for the duration of the execution of some programs and they are trying to find out which are significant. They are ...
vic's user avatar
  • 101
0 votes
0 answers
44 views

How normal is "normal enough"? for a distribution [when using Symbolic Aggregate ApproXimation (SAX)]

I am comparing 4000 time-series (each with 100 points). I would like to apply SAX to discretize these time-series into distinct events in order to make comparisons between/across the time-series My ...
TunaFishLies's user avatar
1 vote
0 answers
18 views

Is there a "proper" way to keep strictly nonnegative data nonnegative when performing PCA, despite centering?

I have a question that came up in my research and I would really appreciate some guidance from someone wise in the ways of dimensionality reduction. I have a dataset of matrices that are strictly ...
Sam Berry's user avatar
1 vote
0 answers
14 views

The meaning of having the same number of principal components as the number of predictive features and using it in my ML model [duplicate]

I have 4 predictive features and 4800 observations. I did PCA fit like this: pca = PCA(n_components=X_train_scaled.shape[1],whiten=True) pca.fit(X_train_scaled) ...
CORy's user avatar
  • 423
4 votes
1 answer
352 views

What are the best ML models for hundreds of binary features (but a non-binary response variable)?

I have hundreds of binary features, resulting in a large binary design matrix (though note that my response variable is not binary). I've tried typical models like logistic regression, KNN, and SVMs ...
xojfqa's user avatar
  • 137
1 vote
1 answer
60 views

In $k$-means, how is it NP-hard if the dimensionality of the data is at least $2$ ($d\geq 2$)?

In $k$-means, how is it NP-hard if the dimensionality of the data is at least $2$ ($d\geq 2$)? Can someone justify or give reasons to this statement? Any guidance would be appreciated.
Maryam Faheem's user avatar
0 votes
0 answers
16 views

Feature selection on models with poor performance

Imagine a $p\gg n$ regression situation in which one wishes to select important features, the goal being to identify a small pool of important predictors and possibly uncover mechanisms of action in ...
JED HK's user avatar
  • 345
1 vote
0 answers
10 views

Metric for target value homogeneity across feature distributions?

I want to find a metric that can quantify target value homogeneity across feature distributions. Without any background knowledge, it is hard to describe exactly what I want. Therefore, I provide an ...
Johnny Tam's user avatar

1
2 3 4 5
22