Questions tagged [dimensionality-reduction]
Refers to techniques for reducing a large number of variables or dimensions spanned by data to a smaller number of dimensions while preserving as much information about the data as possible. Prominent methods include PCA, MDS, Isomap, etc. The two main subclasses of techniques: feature extraction and feature selection.
829
questions
0
votes
0answers
4 views
What is the difference between Feature selection techniques for Classification versus Regression?
Is there any difference between feature selection techniques and methods for Classification, clustering, regression?
For example, features with high colinearity are never preferred in Regression. Is ...
0
votes
1answer
28 views
Can I apply PCA on continuous data and reduce the dimensions and keep categorical data as it is?
I have a dataset which contains 95 highly correlated continuous variables and other 3 categorical variables. I want to reduce the dimension of the data and by that I can deal with correlation as well. ...
0
votes
0answers
27 views
Dimensionality reduction of highly correlated features for predictive maintenance
I have a high dimensional sensor data and many variables are highly correlated. Is it a good idea to get principal components without removing correlated features to reduce dimensions?
My final ...
2
votes
0answers
18 views
How is explained variance in sparse PCA calculated?
Sparse PCA is a technique proposed by Zou et all in this paper. In usual PCA the obtained loadings are orthonormal, and the resulting scores are uncorrelated. However, in sparse PCA you give up these ...
0
votes
0answers
23 views
LDA for binary classification?
My plan was to use LDA as a feature selection method instead of PCA since I have labels. The problem I'm facing is my problem changed to binary classification. So the output of my LDA model is one ...
6
votes
1answer
131 views
What does Sparse PCA implementation in Python do?
I am interested on using sparse PCA in python and I found the sklearn implementation. However, I think this python implementation solves a different problem than the original sparse pca algorithm ...
0
votes
0answers
11 views
Using MCA/PCA together?
If I have a large dataset with continuous, discrete, and categorical data, is it appropriate to use MCA on the categorical features and PCA on the continuous, separately?
I'm preprocessing my data ...
0
votes
1answer
18 views
ICA and orthogonality of Independent Components
In the book by Aapo Hyvärinen, it is shown that:
Where z is the white vector of a data matrix x, s are the IC's and à is the mixing matrix of the whitened data matrix z. My question is: If the matrix ...
2
votes
2answers
31 views
Understand important features in UMAP
I am using a dimensionality reduction algorithm (UMAP) to cluster high-dimensional data.
Particularly, I have ~50000 vectors of dimension ~20000 to visualise. These vectors are highly structured: ...
1
vote
0answers
19 views
Using QR Factorization to improve Sliced Inverse Regression
This code implements Slice Inverse Regression (SIR) in an unusual way. I notice that, when I compare it to the standard algorithm, the modified algorithm does better. By better, I mean that the ...
0
votes
0answers
12 views
Why is the entropy of principal component scores always the same?
Suppose I have a matrix, and I find its principal components. Then I project onto that matrix to get the scores. I then compute the entropy of every row, and it's always the same constant! This is not ...
4
votes
0answers
72 views
Nystrom approximation with inexact/stochastic kernel evaluation
Suppose we have several data points $x_1,\ldots,x_m\in\mathbb R^n$ as well as a positive definite kernel $K(x,y):\mathbb R^n\times\mathbb R^n\to\mathbb R$ that can be written in the form $$K(x,y)=\...
0
votes
0answers
10 views
Does reducing dimensionality of data makes it less linearly separable?
I recently read about kernel trick in SVM that says that mapping data to higher dimensions makes it more linearly separable but can we conversely say that "mapping ...
1
vote
0answers
29 views
What is the connection between fat tailedness of student t distribution and sparsity inducement in lower dimensions in context of t-sne
I have read that the t distribution is a heavy tailed distribution in comparison to Normal distribution. Also some say that heavy tailed distributions help in creating more sparsity.
My question is ...
0
votes
1answer
16 views
What do the matrix (S, U, V) returned by singular value decomposition represent (in terms of variation)?
I believe SVD on a matrix A returns three matrices: U, S, and V.
Let's imagine A is a data matrix with training examples/records/whatever you call them as its rows and attributes as its columns.
I ...
0
votes
0answers
24 views
Feature extraction (LDA) on one hot encoded variables
Background
I am building a machine learning model which identifies variable importance associated with a binary classification problem for guiding data aggregation that a human can then act on. In ...
0
votes
0answers
10 views
Interpretable Unsupervised Clustering of discretized multivariate time-series
Background
Given: a 55 variable (control inputs and sensor data) dataset with 360000 data samples. Data has been discretized with variable step sizes (new data sample created when any one variable ...
0
votes
0answers
6 views
Determining differences in test equipment with dimensional reduction
We use many copies of test stations to test our RF products, and on a regular basis we need to ensure our test stations are all measuring/behaving the same way.
Each sample or test session has ~1000 ...
0
votes
1answer
21 views
Function of Feature Transformation using PCA
I completely understood the math behind PCA. I have a doubt here while calculating the function that will do the transformation. According to the book : Deep Learning by Ian Goodfellow, Yoshua Bengio ...
0
votes
0answers
17 views
Dimensionality reduction preserving K nearest neighbours
I am looking for a dimensionality reduction technique which preserves K nearest neighbours. My input is 800000 2400 dimensional count vectors from sklearn's CountVectorizer and I would like to find ...
0
votes
1answer
17 views
How to pool variables that rarely occur, particularly with respect to survey data
From the text : Multiple Correspondents Analysis by Brigette LeRoux
very infrequent categories of active variables need to be pooled with
others when feasible
The text doesn't explain how this ...
0
votes
0answers
7 views
How many observations need to be in place for multiple correspondence analysis with a particular number of questions/categories
I'm wondering about how many observations need to be in place for a
particular set of questions.
If I have data as follows:
...
1
vote
1answer
22 views
Correct clustering approach for segmenting stores
Domain : Retail
I have a set of stores which I want to cluster into similar stores based on 10 variables: revenue, avg income, market share etc. I took 2 approach:
Approach 1: Given there are 10 ...
0
votes
0answers
10 views
Using dimension reduction techniques for poverty/wealth indicator
I would like to create an indicator/index of a person's wealth (or socio-economic status, SES). I have about 20 variables that are a combination of education, household assets, access to money, and ...
0
votes
0answers
32 views
By what amount can we reduce dimensions using DCT?
I have a numeric dataset of 1000x100(1000 vectors with 100 components) and I'm supposed to reduce its dimensions using PCA and DCT.
I was able to reduce from 100 dimensions to 83 dimensions via PCA ...
0
votes
1answer
59 views
FastICA results not exactly consistent on repetition
I have asked this on stack overflow but couldn't get an answer. I am using the fastICA implementation in R. Example code:
...
1
vote
0answers
34 views
Is the binarization of a multi-class classification worse than a binary classification?
BACKGROUND
Consider $N$ classes $\{C_1, \cdots, C_N\}$ such that the contingency matrix $\mathcal{M}^{\scriptsize(N)}$ produced by some mutli-class classifier $M^{\scriptsize(N)}$ on a test set is $N$...
1
vote
1answer
120 views
How to use PCA to detect outliers?
A PCA will reduce the dimensionality of the original data and construct a subspace generated by eigenvectors of which each represents the (next) highest variance to explain the data.
Let's start at ...
3
votes
1answer
261 views
How can one implement PCA using gradient descent?
I have to implement PCA using gradient descent and stop at convergence. I am not able to find the objective function. I know that
the aim of PCA is to reduce the $n$-dimensional matrix to $k$ ...
2
votes
0answers
22 views
Clustering of very high dimensional data and large number of examples without losing info in dimensions
I'm trying to get a grasp on scalability of clustering algorithms, and have a toy example in mind. Let's say I have around a million or so songs from $50$ genres. Each song has characteristics - some ...
1
vote
0answers
124 views
Invertibility of Random Fourier Features
Is it possible to approximately reconstruct a point $ \mathbf{x} $ in a vector space (say $\mathbb{R}^n $) given it's randomized feature map $ z(\cdot) $ and respective projection $ z(\mathbf{x})$ (in ...
0
votes
0answers
19 views
Applying SVD on dataset with 4 columns
I have a dataset with following format and 200000 rows:
X Y Z A
5608 142 740 1
4533 142 741 2
5620 143 740 0
4732 142 744 1
5500 143 742 1
5514 142 741 2
I am ...
0
votes
0answers
20 views
Comparison/Visualisation of Regression Methods
This question follows this question, in particular @amoeba's clarifying answer and the plot from the SAS documentation included. I'm especially interested in knowing
if $\mathbf{X}, \mathbf{Y}$ are ...
0
votes
0answers
26 views
Can I use the matrix $U$ instead of the matrix $V$ in Principal Component Analysis?
I'm taking Andrew NG's Machine Learning Course and got to the part of Principal Component Analysis. Andrew's implementation of PCA aroused 2 questions for me.
1. Let's say that we have the data ...
-1
votes
1answer
48 views
What does PC1 mean in prcomp output?
I'm having trouble trying to understand the output of the prcomp function from package stats in ...
0
votes
0answers
17 views
Finding PCA-like directions in feature space that maximise sensitivity to a target variable
I have a fairly large space of feature variables in which I want to build a predictor for a target variable. My input dataset for training the predictor are sampled from the space using a mix of log ...
0
votes
0answers
16 views
How to combine PCA scores? [duplicate]
I was wondering if anyone knew how to combine the different PCA scores. In my dataset I have 3 PCs that explain more than 10% of the variation of shape of a bone and each one explains different ...
-1
votes
1answer
35 views
Is it meaningful to reduce dimension from 300 to 100 by PCA?
as I have only learned PCA for a short while, some problems occured when I faced practice. I am willing to accept solution or advice for the following content and great many thanks for anyone's kindly ...
1
vote
2answers
54 views
Why is there a reconstruction loss in PCA with orthonormal eigenvectors?
I've already read How to reverse PCA and reconstruct original variables from several principal components? and I understand conceptually and visually why there has to be a reconstruction loss.
...
0
votes
0answers
10 views
Joining many topic models. How to join similar topics?
I'm doing topic discovery on a large corpus of small texts.
I'm using many topic models, because not having a ground truth, each model covers a different semantic dimension.
So now I ended with more ...
0
votes
1answer
34 views
Saved PCA model produce different result
I'm using PCA to reduce my feature vector dimension. I'm saving its model and transformed output like this:
...
0
votes
0answers
13 views
reduce vector dimensionality for k nearest neighbor search search
I have 5000 512-dimensional vectors. I want to perform cosine similarity Nearest Neighbor on that vectors. I want to store them in DBSM that's why I somehow need to reduce the dimensionality of vector ...
0
votes
0answers
15 views
Treating seasonality in Partial Least Squares forecast
I've been looking for answers on this question but couldn't find concrete solutions so wanted to ask y'all.
I have been playing around trying to forecast an economic/financial-related indicator with ...
7
votes
2answers
176 views
Why is the autoencoder decoder usually the reverse architecture as the encoder?
Every autoencoder architecture I've seen has a similar architecture, mainly that the decoder is just the reverse of the encoder. If the goal of the autoencoder is low-dimensional feature learning, why ...
0
votes
1answer
64 views
Quantitatively assess if independent variables are sufficient to determine dependent variable behaviour
I have the matrix of the independent variables X with dimensions n_samples x m_features and a vector ...
0
votes
0answers
18 views
Will applying SVM in high dimensions (7) with limited training examples (41) likely lead to overfitting?
Right now I have a dataset with 41 training samples (and no testing samples either unfortunately). There are 7 features, but I've been treating the problem as a 2-D problem thus far (in other words, ...
2
votes
2answers
105 views
How to Reduce Number of Variables Before Running Random Forrest or XGBoost
I've simplified the problem I'm working on for this post, so that the focus is on the issue I'm having.
I'm trying to predict if a patient will be diagnosed with arthritis in 2019, based on the ICD-...
0
votes
1answer
85 views
Finding the variables that explain a label through LDA and PCA
I have a dataset of about 200 continuous variables with an also continuous target variable. I want to find those predictors that explain target values that are below 50. To do that I create an ...
0
votes
0answers
7 views
Are there reasonable ways to do bidirectional dimensionality reduction?
Are there any sensible techniques for dimensionality reduction, for example from 20 to 5 dimensions, and then being able (albeit with loss of information) go back from 5 to 20?
Algorithms like t-SNE, ...
2
votes
1answer
50 views
Intuition About Principal Component Directions
I am trying to really get a deep understanding of PCA. From my understanding, a principal component is defined as
$$\mathbf{z}_k = \phi_{1,k} \mathbf{x}_1 + \ldots + \phi_{p,k} \mathbf{x}_p = \mathbf{...
