Refers to techniques for reducing a large number of variables to a smaller number while preserving as much information as possible. Prominent methods include PCA, MDS, Isomap, etc.

learn more… | top users | synonyms

0
votes
0answers
20 views

Help in unsupervised problem formuation for parameter estimation

In the supervised learning problem, the goal is, given a training set, to learn a function $h : X \mapsto Y$ so that $h (x)$ is a “good” predictor for the corresponding value of $y$. If $y$ takes ...
0
votes
0answers
17 views

Will feature reduction help classification, regardless of the algorithm?

I have a data set (originating from text) with p=4000-10000 features (words and/or concepts) and n=2000-4000 obervations. My target value is binary (true/false). I apply different ML algorithms for ...
0
votes
1answer
20 views

Dimensionality reduction of small vectors (image processing)

I have N small floating point vectors of length K (typically, N is in the millions and K=9). I need to compute a lot (millions and millions) of squared euclidean distances between those vectors. It ...
4
votes
1answer
57 views
+50

What is the value of linear dimensionality reduction in the presence of nonlinear alternatives?

From the results I've seen, manifold learning methods seem to generally outperform PCA for complicated, very high-dimensional datasets like images or videos. This makes sense to me, since nonlinear ...
0
votes
0answers
16 views

using variance for dimensionality reduction [on hold]

I have a dataset of 23 variables and 1 class label data. Suppose my variables are (A,B,C,D,E......) Now when i calculate the variance, there are some variables which have very low values.. which state ...
2
votes
0answers
25 views

Why doesn't the hidden state of a neuron network provide better dimension reduction result than original input?

I just read a great post here. I am curious about content of "An example with images" in that post. If the hidden states mean a lot of features of the original picture and getting closer to final ...
0
votes
0answers
13 views

Find similarity or dependency of attributes in a high dimensional dataset [closed]

I have a huge dataset with around 25 variables. All attributes are numerical (continuous) in nature. I want to find the dependency and the structure among these variables. Also what can be the best ...
1
vote
0answers
15 views

how do I measure similarity with canonical correlation analysis?

With canonical correlation analysis for two random vectors $X$ and $Y$, we do SVD on $$(C_XX)^{-1/2} C_XY C_{YY}^{-1/2}$$ to get $U$ and $V$ from the singular vector decomposition, and then define ...
1
vote
2answers
84 views

Modelling with more variables than data points

I'm fairly new to Machine Learning/Modelling and I'd like some background to this problem. I have a dataset where the number of observations is $n<200$ however the number of variables is $p\sim ...
2
votes
0answers
24 views

t-SNE versus MDS

Been reading some questions about t-SNE (t-Distributed Stochastic Neighbor Embedding) lately, and also visited some questions about MDS (Multidimensional Scaling). They are often used analogously, ...
0
votes
0answers
9 views

How to determine parameters for t-SNE for reducing dimensions?

I am very new to word embeddings. I want to visualize how the documents are looking after learning. I read that t-SNE is the approach to do it. I have 100K documents with 250 dimensions as size of the ...
0
votes
0answers
7 views

Comprehensive list of dimensional reduction methods?

I was wondering if something like this existed on the site. I found this https://e-reports-ext.llnl.gov/pdf/240921.pdf which mentions PCA LA and ICA but I was wondering if there was more, for example ...
2
votes
0answers
29 views

How does Principal Coordinate Analysis (PCoA) work, as compared to PCA? [duplicate]

I am familiar with PCA from Making sense of principal component analysis, eigenvectors & eigenvalues where you either normalize the data (to standard normal or ...
3
votes
2answers
82 views

Projecting data on a sphere

I am used to working with PCA, tSNE, LLEs... They all do a great job projecting the data on a plane (or on linear subspaces of $\mathbb{R}^n$). Is there any other embedding technique that projects the ...
0
votes
0answers
21 views

Synthetic data generating for dimension reduction algorithm testing

I would like to ask if somebody can help me with the following problem. I would like to generate synthetic data for dimension reduction algorithm testing. Specifically, I would like to have for ...
0
votes
0answers
24 views

The total variance explained by a system

We start with a set of objects $V=\left\{ { v }_{ 1 },{ v }_{ 2 },\dots ,{ v }_{ n } \right\} $ and a metric $f\left( { v }_{ a },{ v }_{ b } \right) \rightarrow d$, where $0 \le d \le 1, d \in \...
0
votes
0answers
10 views

How to tune hyperparameters for LLE?

I'm running LLE using Scikit-Learn (with the LocallyLinearEmbedding class), but there are a few hyperparameters and I would like to use grid search with cross-...
0
votes
0answers
11 views

Is centroid came from normalized or reduced data still valid?

I want to incorporate clustering to simplify data and speed up classification execution time. Let's say I have data like this : ...
1
vote
1answer
31 views

Bottleneck building block in Residual learning networks

I am wondering about how 1x1 convolution can be used to change the dimensionality of feature maps in a residual learning network. Here the top 1x1 convolution changes the feature map size from 256 ...
2
votes
2answers
68 views

Linear “self” regression, terminology and references?

Suppose that $X_i, i=1,\ldots,n$ are some random variables. I'd like to do multiple linear regression to learn to predict any of these variables from the others. My model for the reconstructed ...
1
vote
0answers
40 views

Dimensionality reduction for multivariate time series

I have a data set including 25 variables $(x_{1,t},\dotsc,x_{25,t})$ at each time $t$ and all of this group is repeated through time. I want to explore the relationship between these variables through ...
0
votes
0answers
16 views

How can I have more factors than there are variables

I've come to learn about factor analysis as mainly a dimensionality reduction tool. However, can we also use the factor analysis method to have more factors than there are variables? If so, what would ...
0
votes
0answers
16 views

Problem with syntetic data generating for Probabilistic PCA and Factor Analysis (FA) comparison - methodology

I am trying to understand a short example related to dimension reduction from python scikit-learn.org official documentation for long time and unfortunately I am not successful. I don't have problems ...
12
votes
4answers
2k views

Does “curse of dimensionality” really exist in real data?

I understand what is "curse of dimensionality", and I have done some high dimensional optimization problems and know the challenge of the exponential possibilities. However, I doubt if the "curse of ...
0
votes
1answer
72 views

Partitioning training data for dimension reduction and classification

Let's say I want to test the performance of my dimension reduction + classification pipeline. To do this, I will use k-fold cross validation. I know that performing dimension reduction on the complete ...
0
votes
0answers
28 views

Does it make sense to do PCA before kernel regression?

I have a set of features extracted from the same samples and I'm learning a kernel ridge regression. Now, especially for feature fusion, reducing the number of features before combining them seems ...
0
votes
0answers
35 views

Variable selection in logistic regression model

Imagine a data set with approximately 100 variables and 5000 cases. The outcome is a two-level factor. All variables are factors, most of them three levels (yes, no, or indifferent). After building a ...
2
votes
3answers
132 views

Autoencoder wrongly removes objects from images

For a university project, we want to use reinforcement learning from raw camera input to teach a robot to hit a ball. It works when we detect the ball and feed that to the algorithm, so raw data is ...
0
votes
0answers
10 views

Embedding categorical values

Is it possible to perform embedding of a dataset that contains only categorical values e.g. n f1 f2 f3 1 2 1 3 2 2 2 3 3 1 3 2 4 1 4 1 where n is an object and f1, f2, f3 are ...
0
votes
0answers
9 views

Identification in a gaussian two factor model

I am working with a Gaussian two factor model: $$ X_i = \beta_iZ_1+\gamma_iZ_2+\varepsilon_i, \space i = 1,2,...,n $$ where $Z_j\sim N(0,1), \space Z_1 \perp Z_2 $ and $\varepsilon_i\space iid\space ...
1
vote
0answers
8 views

How flexible are autoencoders for non-linear dimensionality reduction?

I've been starting to play around with autoencoders for feature extraction and dimensionality reduction, and am wondering how critical input feature definitions are for success. For example, some of ...
0
votes
0answers
40 views

Can Factor Analysis on Mixed Data be treated like a PCA?

I wanted to do something equivalent to a PCA on a mixed data set containing categorical variables and continuous numerical predictor variables which are normally distributed but measured in very ...
1
vote
0answers
20 views

Function Separation

I have a lot of measurements of a function f(x,y) (lets say a lot more than a million measurements) at known values of the vectors x and y. The inputs to the function are the vectors x and y and the ...
2
votes
3answers
59 views

2 features and 2 principal components

Whats the difference between 2 features and 2 principal components? I know what a PCA is, I just have the following problem: If my data has 2 features, the PCA will produce 2 components. So why does ...
-1
votes
1answer
38 views

When can clustering be used for dimensionality reduction? [closed]

Can a clustering method be used for dimensionality reduction? I though the answer would be that the cluster numbers can act as the synthetic reduced dimension -- but the other day a friend had a more ...
0
votes
1answer
151 views

PCA vs FA vs ICA for dimensionality reduction in questionaire data

I am trying to identify personality traits underlying the multidimensional data from a questionnaire. In more abstract terms this means reducing the dimensionality of my data from N-dimensional (where ...
3
votes
1answer
86 views

Do I need to run PCA over all predictors in a regression model? Can I run it only over the continuous ones?

I'm looking at the Lending Club data from Kaggle and I'm just building a pretty simple model to predict defaults. The data has a large amount of both continuous and categorical variables (I have ...
1
vote
0answers
37 views

How to determine time complexity of EM algorithm of probabilistic PCA?

I was studying probabilistic PCA from Bishop's book. There an EM algorithm is provided to calculate principal subspace: Here $\mathbf M$ is $M\times M$ matrix, $\mathbf W$ is $D\times M$ matrix ...
2
votes
0answers
73 views

If almost all variance is explained by the few first principal components, what can we say about the dataset?

What can we say about a dataset if we apply PCA and observe that there is a high percentage of variance in the first principal component(s)? Can we say that this dataset has linear structure? Can we ...
1
vote
1answer
44 views

How does one visualize the self-organizing map of $n$-dimensional data

I have a data set consisting from $7$-dimensional data points. I want to produce a self-organizing map for this data to see how my data is clustered. I have been reading some tutorials from the ...
1
vote
1answer
162 views

Why convert categorical data into numerical using one hot encoding

I don't have very strong statistical background, and I'm new in data science... Now, I am practicing PCA (Principle Component Analysis) for dimension reduction. This tutorial looks very complete, but ...
6
votes
1answer
180 views

Is PCA a non-linear transform?

In the article Relative Information Loss in the PCA, the authors make, at some point (in the introductory section), the following statement: In case the orthogonal matrix is not known a priori, ...
0
votes
1answer
51 views

Negative coefficients for ordinal logistic regression in R

I am trying to model my dependent variable (ordinal - three levels) using a set of independent variables (5 ordinal and 10 numeric). I am using lrm function in "rms"...
1
vote
0answers
10 views

Feature extraction from data in the form of many manifolds, in hierarchial structure and various dimensions

Is there a known feature extraction method which was developed to cope with data that satisfies the following assumptions?: The data points are real valued vectors in ...
1
vote
0answers
23 views

How to choose an optimal dimension reduction factor in LSA processing

I'm performing a K-Means clustering on a 400.000 text dataset. After eliminating useless chars and removing stopwords, I get a dictionnary size of around 42000 words. So before doing the clustering, ...
0
votes
0answers
13 views

Filter Feature Selection approaches for continuous variables?

I've noticed that correlation-based filtering for selecting features in high dimensional data require discretization of continuous variables, like e.g. Fast Correlation-based Filtering or regular CFS. ...
9
votes
1answer
202 views

What is the connection between partial least squares, reduced rank regression, and principal component regression?

Are reduced rank regression and principal component regression just special cases of partial least squares? This tutorial (Page 6, "Comparison of Objectives") states that when we do partial least ...
2
votes
1answer
29 views

Self Organizing Maps: How is the location computed and updated?

I have read other similar questions on here, but I am still unsure how SOM deals with the positions/locations of the neurons. Say that the input space is N-dimensional. I initalise some weights, and ...
0
votes
1answer
33 views

Choosing how many factors to retain based on parallel analysis and on a scree plot without an elbow

When I realize the Factor Analysis (I have 16 items), the PCA says I have 5 factors. But in the scree plot there is no elbow at all, just a decreasing line, that makes me think maybe I shouldn't be ...
1
vote
0answers
10 views

Understanding the bound given by Johnson–Lindenstrauss lemma

This was a question I asked in the mathematics site. Though no one replied but I have figured it out. But new questions arise. Former Question: Here I choose to use the statement made by S.Dasgupta: ...