Questions tagged [dimensionality-reduction]
Refers to techniques for reducing a large number of variables or dimensions spanned by data to a smaller number of dimensions while preserving as much information about the data as possible. Prominent methods include PCA, MDS, Isomap, etc. The two main subclasses of techniques: feature extraction and feature selection.
0
votes
0answers
5 views
is random projection a linear or non-linear feature extraction method?
The dimensionality reduction has two different types: feature selection and feature selection. As far as i know, the random projection cannot be a feature selection method. Therefore, is it a linear ...
0
votes
0answers
36 views
Determining the Direction of Eigenvectors in PCA [duplicate]
I'm using R to get the principal components for several datasets.
An example result, using prcomp yields:
...
1
vote
1answer
101 views
Can t-SNE help feature selection?
I'm training a fully connected feed forward neural network for regression.
Given one training example $(x_i, y_i)$, I need to convert the raw representation $x_i$ into an invariant representation $...
7
votes
2answers
671 views
What is the problem with $p > n$?
I know that this is the solving system of linear equation problem.
But my question is why it is a problem the number of observation is lower than the number of predictors how can that thing happen?
...
0
votes
0answers
17 views
What distinguishes the different dimensions of t-Distributed Stochastic Neighbor Embedding (tSNE)
I understand conceptually how the tSNE algorithm works in one dimension. But I am confused how this works for 2 or more dimensions (let us call the number of dimensions of the lower dimensional space ...
0
votes
0answers
14 views
Feature reduction of Biological time series signals
I have a data set of biological signals (PSG signals); the dimension of the signals is high (850 features for each sample). I am looking for the best way to reduce the dimensionality of the signals. ...
0
votes
1answer
21 views
Minimum Input Dimension for Autoencoder Neural Network
Model: Assume we want to learn patterns using an autoencoder neural network. In the simplest case, such a network is "shallow" with 1 hidden layer, takes a $d$-dimensional numerical input vector $x$, ...
1
vote
0answers
28 views
What is connection between Pearson correlation coefficient and proportion of variance explained in PCA?
PCA procedure includes SVD of Covariance matrix. Based on eigenvalues we can find a proportion of variance explained by related Principal Components (eigenvectors).
...
0
votes
0answers
19 views
How to find orthogonal projection of a d-dimensional point, d>2 on Fisher's discriminant?
We know that the number of Fisher's discriminant to find for classifying data in given dimension d into output classes K, is min(K-1,d)
Using the iris dataset as an example, we know that d=4 (petal ...
2
votes
0answers
16 views
Searching intersection of elements in subsets with approximation
Task:
Having a large number of transactions that consists of distinct elements from one large set $S$ I need to find transactions in which items have intersection with more than 20% of items in ...
0
votes
0answers
8 views
Nipals and prcomp differences and data size relationship [closed]
I'm pretty new in R and data analysis and I found a strange behaviour which I wanted to discuss with you.
I had a 486 observation of 5000+ variables and I want to cluster them with Mclust. Each value ...
8
votes
4answers
247 views
Is “random projection” strictly speaking not a projection?
Current implementations of the Random Projection algorithm reduce the dimensionality of data samples by mapping them from $\mathbb R^d$ to $\mathbb R^k$ using a $d\times k$ projection matrix $R$ whose ...
-1
votes
1answer
37 views
PCA influence of duplicates
I am using sklearn IPCA decomposition and surprised that if I delete duplicates from my dataset, the result differs from the "unclean" one.
What is the reason? As I think, the variance is the same.
...
0
votes
0answers
43 views
Using UMAP or other non-linear dimension reduction techniques on response variables prior to learning?
Background
Suppose you have a training set where the response measurements are some $N$-dimensional vectors of related measurements - in my specific case, they happen to be cell viability scores for ...
1
vote
0answers
35 views
Log-transformation of Compositional Data
I am dealing with compositional data, in a high dimension.
Each sample I have behaves like:
$$ {S}^D=\left\{\mathbf{x}=[x_1,x_2,\dots,x_D]\in\mathbb{R}^D \,\left|\, x_i>0,i=1,2,\dots,D; \sum_{i=1}...
0
votes
0answers
14 views
Implementation of Compressed Regression (examples?)
I found several articles discussing compressed regression, whether in the bayesian framework or ordinary LS/GLM (1, 2, 3 and others).
The idea seems simple, the model becomes
$$ Y = \Phi X \beta + \...
0
votes
1answer
34 views
Terminology: the “t-distribution” in t-SNE?
I am trying to learn the dimensionality reduction using t-SNE technique. After some videos and explanation I understood the idea behind it.
But I am not getting where the "t-distribution" is used ...
1
vote
0answers
27 views
t-SNE for finding nearest neighbors
I had a question about dimensionality reduction for finding nearest neighbors and was hoping someone could help me out here.
Suppose I have good features for images, say penultimate layer features ...
1
vote
0answers
51 views
How to reduce sample size in predictive model?
I am working on a regression model, which purpose is solely to predict new data. My training dataset contains around 200 measurements and with a best subset regression i was able to find good models ...
-1
votes
0answers
11 views
Transforming Dimensions of PCA/MCA/FAMD/MFA to strict positive values
I´m curious to know if the outcome of a given principal component analysis or other related methods can be transformed to strict positive values. I know every dimension is represented by a continuum ...
1
vote
1answer
32 views
Interpreting t-SNE dimensions in terms of the original features [duplicate]
I have a dataset of 223 multilabels. I'm sure some of those labels are correlated so I want to merge them. For that I'm using sklearn tSNE to reduce the dimensions.
How can I "describe" the new ...
2
votes
0answers
30 views
Intuition behind perplexity parameter in t-SNE
While reading Laurens van der Maaten's paper about t-SNE we can encounter the following statement about perplexity:
The perplexity can be interpreted as a smooth measure of the effective number of ...
1
vote
2answers
40 views
Dimensionality reduction: include labels?
Lets say I have something like the iris dataset, the columns are petal length, petal width, ...
0
votes
1answer
60 views
Applying PCA - First two components explain low variance but have high data separation when plotting
Applying PCA on a set of documents gives strange results in terms of the variance explained by the PCs vs the data separation I'm having when plotting the first two principle components.
Details:
...
0
votes
1answer
27 views
Does it make sense to run lasso to select features for neural network training?
I want to train a neural network for regression.
$$\Bbb R^{2800} \rightarrow \Bbb R^{1}$$
The dimension of feature vectors is $2800$. The figure is an illustration of one of the feature vectors.
...
0
votes
1answer
64 views
Dimensionality reduction with least distance distortion
Question: Could I find a dimensionality reduction algorithm without or with minimal distance (cosine) distortion?
Background: I would like to visualize in 2D a sample of news texts for which I also ...
2
votes
0answers
47 views
Differences between t-SNE and SOM
I have some high dimensional data and I want to reduce it to 2 dimensions for visualization. The goal is to color the points in this 2D space to see whether there is any clustering due to different ...
0
votes
1answer
23 views
Doubt regarding PCA
I have 5 different independent variables, lets name 1 to 5. The 3rd IV has 10 sub-variables under it and 4th IV has 11 sub-variables in it. Whereas other 3 IV's have just two sub-variables (...
0
votes
0answers
24 views
Principal components: Can I interpret PCA as essentially a change of basis
I was hoping that someone could simply validate or correct my interpretation of Principal Components Analysis. There are a lot of questions on this site about Principal Components analysis--some ...
0
votes
0answers
5 views
How to cluster and reduce dimensionality of Barthel scale data
I 'd like to cluster subjects with their Barthel scale. It looks like below:
Since the Barthel scale data is an ordinal scale and a likert-type rating data (given number of points assigned to each ...
0
votes
2answers
45 views
How to find the minimum set of vectors?
I have a set of vectors. For simplicity, suppose there are $n=20$ vectors and each has $p=5$ elements.
These vectors are generated from some experiments, and so it is not very apparent how they are ...
-2
votes
1answer
40 views
Example for Principal Component Analysis
Where principal component analysis can potentially be used ?
some examples with some explanation would be great
0
votes
1answer
38 views
What does ICA return?
I am confused with ICA. With PCA I understood that it always gives the components with maximum variance. What does ICA return? Does it return components with maximum independence? How to find best ...
0
votes
1answer
16 views
Self-organizing map dimension
I just have a question on the chosen dimension of Self-organizing map.
Typically, an SOM has a dimension of 2 or 3 but rarely larger than that. Is there a particular reason why this is the case?
0
votes
0answers
18 views
Gaussianity and Whitening in ICA - The feeling and intuition behind it
I understand what ICA does at a high level but in the cocktail party problem context. All the examples, articles I have read take a similar problem to explain ICA where the aim is to derive the ...
0
votes
0answers
9 views
optimality of LDA dimensionality reduction
typically when you do dimensionality reduction using LDA, you select $n_{class}-1$ vectors with largest eigenvalues as discriminants given the fact that you only need $n-1$ dimensions to classify $n$ ...
1
vote
1answer
104 views
How do I get the density of a region in a vector space?
I have a simple problem, which I think must have an easy solution.
I have a vector space say with a 1000 dimensions for each vector.
Now, I have a large number of sample vectors from this vector ...
0
votes
0answers
42 views
Dimensionality reduction for multivariate time series data?
I'm trying to perform dimensionality reduction on a multivariate time-series data set which gives information on the medical history of a group of 12000 patients. The data contains 19 different ...
1
vote
0answers
34 views
Linear Discriminant Analysis as Dimensionality Reduction very sensitive to Training Set size
I'm working with supervised classification of object-based satellite imageryand currently investigate different dimensionality reduction methods on their suitability to this application. As part of my ...
1
vote
0answers
28 views
What kind of PCA is best for my data?
I'm using sklearn to reduce the dimensionality of an object.This scan is 3D, with shape [15,15,15] and contains mostly zeros, altough the non-zero values are usally clustered together in some groups.
...
0
votes
1answer
37 views
PCA Dimension Reduction for Cox Proportional Hazard Model
I am trying to figure out if it is possible and appropriate to use PCA to reduce the dimensions of my dataset? A little background - I have survival data with 8 covariates. I have run a Pearson's ...
0
votes
0answers
20 views
Variability in categorical variables while reducing dimensionality
If a dataset has a variable with categorical values (A, B, C, D, E, F), would converting them to numeric and breaking them down into separate columns (for example, column A that will have 1s for all ...
2
votes
0answers
27 views
PCA: mean of marginal distribution of high-dimensional vector
Consider the following probabilistic model:
$$p(x) = \mathcal{N}(0, I_d), \ x \in \mathbb{R}^d$$
$$p(y|x) = \mathcal{N}(Wx + \mu, \sigma^2I_D),
\ y, \mu \in \mathbb{R}^D, W \in \mathbb{R}^{D\times d}...
0
votes
0answers
15 views
Performing PCA on properly scaled data [duplicate]
I would like to perform PCA on a dataset, however, not all of the data is in the same scale. Some variables are Height, Weight, Age, while others are Dribbles per Game, Shots per Game, Blocks per Game....
1
vote
2answers
79 views
Non-negative matrix factorization (NMF) on mixed data using 1-hot encoding
From a standpoint of interpretation, can I use NMF on one-hot encoded categorical data for dimension reduction? I have mixed data and was thinking about one-hot encoding the categorical features and ...
5
votes
2answers
397 views
Why with two classes, LDA gives only one dimension?
I am working with dimensionality reduction algorithms. Linear Discriminant Analysis (LDA) is a supervised algorithm that takes into account the class label (which is not the case of PCA for example). ...
1
vote
0answers
43 views
Comparing PCA representations between low-pass and high-pass filtered time-series data
I am currently trying to reduce the number of variables I input into a vector autoregressive (VAR) model. For those that don't already know, VAR models are used on time-series data. My primary concern ...
1
vote
0answers
15 views
Bag of Visual Words: is feature extraction even needed?
I'm currently implementing a BoVW as part of my lab project. The steps the algorithm used are as follows:
spliting all photos into patches
cluster these pathces using K-means based on pixel values of ...
1
vote
1answer
37 views
Regression with multidimensional output variable Y
Say we have an $N \times q$ matrix $Y$ with $N>q$.
Also, we have an $N \times p$ data matrix $X$.
We are interested in a model of $Y = X \times W + \epsilon$, where $W$ is a $p \times q$ matrix ...
0
votes
1answer
21 views
Clustering by same random projection
I have $N$, $1024$-dimensional vectors. I want to cluster them by some similarity. Given the high dimensionality, standard metrics won't work. I tried a few Approximate Nearest Neighbor ...
