# Tagged Questions

Refers to techniques for reducing a large number of variables to a smaller number while preserving as much information as possible. Prominent methods include PCA, MDS, Isomap, etc.

20 views

### Help in unsupervised problem formuation for parameter estimation

In the supervised learning problem, the goal is, given a training set, to learn a function $h : X \mapsto Y$ so that $h (x)$ is a “good” predictor for the corresponding value of $y$. If $y$ takes ...
17 views

### Will feature reduction help classification, regardless of the algorithm?

I have a data set (originating from text) with p=4000-10000 features (words and/or concepts) and n=2000-4000 obervations. My target value is binary (true/false). I apply different ML algorithms for ...
20 views

### Dimensionality reduction of small vectors (image processing)

I have N small floating point vectors of length K (typically, N is in the millions and K=9). I need to compute a lot (millions and millions) of squared euclidean distances between those vectors. It ...
57 views
+50

### What is the value of linear dimensionality reduction in the presence of nonlinear alternatives?

From the results I've seen, manifold learning methods seem to generally outperform PCA for complicated, very high-dimensional datasets like images or videos. This makes sense to me, since nonlinear ...
16 views

### using variance for dimensionality reduction [on hold]

I have a dataset of 23 variables and 1 class label data. Suppose my variables are (A,B,C,D,E......) Now when i calculate the variance, there are some variables which have very low values.. which state ...
25 views

### Why doesn't the hidden state of a neuron network provide better dimension reduction result than original input?

I just read a great post here. I am curious about content of "An example with images" in that post. If the hidden states mean a lot of features of the original picture and getting closer to final ...
13 views

### Find similarity or dependency of attributes in a high dimensional dataset [closed]

I have a huge dataset with around 25 variables. All attributes are numerical (continuous) in nature. I want to find the dependency and the structure among these variables. Also what can be the best ...
15 views

### how do I measure similarity with canonical correlation analysis?

With canonical correlation analysis for two random vectors $X$ and $Y$, we do SVD on $$(C_XX)^{-1/2} C_XY C_{YY}^{-1/2}$$ to get $U$ and $V$ from the singular vector decomposition, and then define ...
84 views

10 views

### How to tune hyperparameters for LLE?

I'm running LLE using Scikit-Learn (with the LocallyLinearEmbedding class), but there are a few hyperparameters and I would like to use grid search with cross-...
11 views

### Is centroid came from normalized or reduced data still valid?

I want to incorporate clustering to simplify data and speed up classification execution time. Let's say I have data like this : ...
31 views

### Bottleneck building block in Residual learning networks

I am wondering about how 1x1 convolution can be used to change the dimensionality of feature maps in a residual learning network. Here the top 1x1 convolution changes the feature map size from 256 ...
68 views

### Linear “self” regression, terminology and references?

Suppose that $X_i, i=1,\ldots,n$ are some random variables. I'd like to do multiple linear regression to learn to predict any of these variables from the others. My model for the reconstructed ...
40 views

### Dimensionality reduction for multivariate time series

I have a data set including 25 variables $(x_{1,t},\dotsc,x_{25,t})$ at each time $t$ and all of this group is repeated through time. I want to explore the relationship between these variables through ...
16 views

### How can I have more factors than there are variables

I've come to learn about factor analysis as mainly a dimensionality reduction tool. However, can we also use the factor analysis method to have more factors than there are variables? If so, what would ...
16 views

### Problem with syntetic data generating for Probabilistic PCA and Factor Analysis (FA) comparison - methodology

I am trying to understand a short example related to dimension reduction from python scikit-learn.org official documentation for long time and unfortunately I am not successful. I don't have problems ...
2k views

### Does “curse of dimensionality” really exist in real data?

I understand what is "curse of dimensionality", and I have done some high dimensional optimization problems and know the challenge of the exponential possibilities. However, I doubt if the "curse of ...
72 views

### Partitioning training data for dimension reduction and classification

Let's say I want to test the performance of my dimension reduction + classification pipeline. To do this, I will use k-fold cross validation. I know that performing dimension reduction on the complete ...
28 views

### Does it make sense to do PCA before kernel regression?

I have a set of features extracted from the same samples and I'm learning a kernel ridge regression. Now, especially for feature fusion, reducing the number of features before combining them seems ...
35 views

### Variable selection in logistic regression model

Imagine a data set with approximately 100 variables and 5000 cases. The outcome is a two-level factor. All variables are factors, most of them three levels (yes, no, or indifferent). After building a ...
132 views

### Autoencoder wrongly removes objects from images

For a university project, we want to use reinforcement learning from raw camera input to teach a robot to hit a ball. It works when we detect the ball and feed that to the algorithm, so raw data is ...
10 views

### Embedding categorical values

Is it possible to perform embedding of a dataset that contains only categorical values e.g. n f1 f2 f3 1 2 1 3 2 2 2 3 3 1 3 2 4 1 4 1 where n is an object and f1, f2, f3 are ...
9 views

162 views

### Why convert categorical data into numerical using one hot encoding

I don't have very strong statistical background, and I'm new in data science... Now, I am practicing PCA (Principle Component Analysis) for dimension reduction. This tutorial looks very complete, but ...
180 views

### Is PCA a non-linear transform?

In the article Relative Information Loss in the PCA, the authors make, at some point (in the introductory section), the following statement: In case the orthogonal matrix is not known a priori, ...
51 views

### Negative coefficients for ordinal logistic regression in R

I am trying to model my dependent variable (ordinal - three levels) using a set of independent variables (5 ordinal and 10 numeric). I am using lrm function in "rms"...
10 views

### Feature extraction from data in the form of many manifolds, in hierarchial structure and various dimensions

Is there a known feature extraction method which was developed to cope with data that satisfies the following assumptions?: The data points are real valued vectors in ...
23 views

### How to choose an optimal dimension reduction factor in LSA processing

I'm performing a K-Means clustering on a 400.000 text dataset. After eliminating useless chars and removing stopwords, I get a dictionnary size of around 42000 words. So before doing the clustering, ...
13 views

### Filter Feature Selection approaches for continuous variables?

I've noticed that correlation-based filtering for selecting features in high dimensional data require discretization of continuous variables, like e.g. Fast Correlation-based Filtering or regular CFS. ...
202 views

### What is the connection between partial least squares, reduced rank regression, and principal component regression?

Are reduced rank regression and principal component regression just special cases of partial least squares? This tutorial (Page 6, "Comparison of Objectives") states that when we do partial least ...
29 views

### Self Organizing Maps: How is the location computed and updated?

I have read other similar questions on here, but I am still unsure how SOM deals with the positions/locations of the neurons. Say that the input space is N-dimensional. I initalise some weights, and ...