Dimensionality reduction refers to techniques for reducing many variables into a smaller number while keeping as much information as possible. One prominent method is [tag pca]

learn more… | top users | synonyms

1
vote
0answers
26 views

Order of preprocessing steps in a binary classification problem

I have these stages (ordered) for preprocessing in my binary classification problem. Dividing data based on criteria (class1 and class2 databases) Outlier ...
2
votes
0answers
41 views

Using principal component analysis to reduce dimensions of data in R [closed]

I have a dataset which includes 4 separate measures of intelligence. To simplify my analysis, I wanted to express them as "g" a variable based on the shared variation of the 4 measures. A paper I read ...
-1
votes
0answers
11 views

regression. sufficient dimension reduction [duplicate]

in sufficient dimension reduction technique, we image predictors on central subspace. this procedure product new predictors for us that are linear combination of original predictors. in addition, new ...
2
votes
1answer
39 views

Linear Discriminant Analysis and non-normal distributed data

If I understand correctly, a Linear Discriminant Analysis (LDA) assumes normal distributed data, independent features, and identical covariances for every class for the optimality criterion. Since ...
3
votes
1answer
55 views

Reference for dimension reduction techniques

This is a follow-up question to Is PCA appropriate for comparing subsets of panel data?. It turns out that, yes, PCA is appropriate. But there are also many other ways to reduce n-dimensional data to ...
7
votes
3answers
278 views

Why Python's scikit-learn LDA results are different from LDA in R or a step-by-step approach

I was using the Linear Discriminant Analysis (LDA) from the scikit-learn machine learning library (Python) for dimensionality reduction and was a little bit curious about the results. I am wondering ...
0
votes
0answers
32 views

Can PCA scores be used as dependent variable?

I am working on a research project where I have several questions from a survey data that measures the same underlying quantity (my dv), possibly each with some measurement error. I was thinking about ...
2
votes
2answers
34 views

Dimensionality reduction (PCA) for plotting text documents on a graph

I have 50 text documents There are 500 possible words, after a stop list has been applied My term/document sparse matrix is therefore 50x500 I'd like to cluster these documents. One easy way to do ...
0
votes
0answers
21 views

Rotation of Mean Centred Variables in Principle Components Analysis

I'm looking to manually (Excel) perform PCA without any statistical packages such as R, but having trouble understanding how to rotate the original variables to find the maximum variance for the new ...
3
votes
1answer
50 views

When is it appropriate to use PCA as a preprocessing step?

I understand that PCA is used for dimensionality reduction to be able to plot datasets in 2D or 3D. But I have also seen people applying PCA as a preprocessing step in classification scenarios where ...
0
votes
1answer
31 views

How to use SVD for dimensionality reduction to reduce columns specifically?

My original data has many more columns (features) than rows (users). I'm trying to reduce the features of my SVD (I need all of the rows). I found one method of doing so in a book called "Machine ...
0
votes
0answers
16 views

Reconstructing a vector after projection

Suppose one has a matrix of data $X$, which is $n$ observations by $p$ dimensions. Let $P_\perp$ be a projection onto some $k<p$ dimensional subspace. Suppose one computes the principal direction ...
3
votes
2answers
144 views

Best practice for dimensionality reduction using Principal Component Analysis (PCA) and/or Linear Discriminant Analysis (LDA)

Assume I have a dataset for a supervised statistical classification task, e.g., via a Bayes' classifier. This dataset consists of 20 features and I want to boil it down to 2 features via ...
1
vote
0answers
32 views

Creating classification features from wavelet transformed time series

I'm interested in using a wavelet transform, Haar for example, to create classification variables from time series data to use in logistic regression. Simple example. Let's say I'm trying to predict ...
4
votes
2answers
102 views

How to reduce the dimension of $10^8$ vectors

I have $10^8$ vectors in $1000$ dimensions each. I would like to drastically reduce their dimensions. However PCA seems computationally infeasible. Are there near linear time methods to do ...
4
votes
1answer
147 views

How to distance and to MDS-plot objects according their complex shape

Suppose I have four basal forms of signal (blue, purple, red, green). I also have created transition forms between each other. If you carefully look on the picture below, you can see that for example ...
2
votes
1answer
28 views

Method to compare ratings from multiple different sources with missing data

I want a method to compare ratings from multiple sources and find a single measure that best reflects all the ratings. To give a specific example, let's call it "The fellowship review committee ...
0
votes
1answer
92 views

PCA on Binary Data

I having binary data set (yes/no), so can I apply PCA on that. Is it mathematically correct to do that. In my opinion Binary variable can only be subjected to logical operations, so how it can be ...
0
votes
0answers
11 views

Quality of NER classification decreases dramatically when sliding window transformation is performed

I'm writing a NER classificator now. That performs quite well even without window transformation (~80% F1-score for non-English language is quite well, AFAIK), but the strange thing is that when I ...
0
votes
1answer
59 views

PCA-based dimensionality reduction when the number of data points is less than the number of variables

Assume we are given a $p \times n$ (variables $\times$ data points) data matrix $X$, with $p > n$ (i.e. more variables than data points). Performing PCA on such a matrix yields an $n \times n$ ...
4
votes
2answers
159 views

Why does PCA maximize variance of the projection?

Christopher Bishop writes in his book (Pattern Recognition and Machine Learning) a proof, that each consecutive principal component maximizes the variance of the projection to one dimension, after the ...
0
votes
0answers
43 views

Machine learning from implied variables

I have a situation where we are detecting anomalies based on data implied from the table data. As an example, I have data on registered individuals spending time on the portal. Based on this, I have ...
0
votes
0answers
74 views

SVD application for a Boolean sparse Matrix

Basically, I am trying to have a recommender system based on SVD for a Boolean utility matrix. ie If at all some entries are present in the utility matrix, they will be 1 (I made it pseudo-implicit ...
2
votes
2answers
57 views

Which distribution has the larger variability? Is variance absolute, or relative to the mean?

I apologize in advance, I'm new to statistics. I have a large (millions) dataset (the US Census American Community Survey) with 286 attributes. I've calculated the mean, variance and standard ...
1
vote
1answer
58 views

Number of components in PCA

I believe I have a problem understanding PCA: I would like to use this technique to reduce the number of features of my problem. I originally have 10,000 features and 500 samples. However, the use of ...
1
vote
1answer
100 views

Principal component analysis with known uncertainty varying across both samples and variables

I want to do dimensionality reduction on a data set $X_{ij}$. In this case, $i$ indexes samples and $j$ indexes a large number of variables (densities at different locations in space). The units of ...
1
vote
1answer
65 views

PCA followed by LDA: dimension reduction strategy

I have a high dimensional dataset (n*p: 30 * 100) which I want to use as an testing dataset to build a two group classifier (LDA or QDA). I've read that you can do PCA to do an dimension reduction of ...
2
votes
2answers
81 views

Does fastICA require PCA to run at first?

I reviewed an application based paper saying that applying PCA before applying ICA (using fastICA package). My question is that does ICA (fastICA) requires PCA to run at first? This paper mentioned ...
0
votes
1answer
40 views

Image vectorization in matlab

I need to convert images into vectors (image vectorization). I have 165 images in total, divided into 15 subjects of 11 images each. Following is code i have written to convert images into vectors. ...
1
vote
0answers
33 views

How do you explain dimension-reduction with statistics?

With statistics: how would you explain the dimensionality reduction and dimensionality addition? Like the conversion of a color picture to gray space so that a color blind person could more easily ...
7
votes
4answers
206 views

Non-orthogonal technique analogous to PCA

Suppose I have a 2D point dataset and I want to detect the directions of all the local maxima's of variance in the data, for example: PCA does not help in this situation as it is an orthogonal ...
2
votes
1answer
31 views

Selecting features manually and proving the rest are redundant

I'm working with a gesture dataset, where each gesture has a variable number of frames, and each frame has the 3d position of 20 joints, so that each gesture is represented by a matrix of size frames ...
0
votes
0answers
20 views

Subspace clustering with random transformation

One approach for clustering a high dimensional dataset is to use linear transformation, and the most common approaches are PCA and random projection (where random projection arises from the ...
0
votes
0answers
80 views

PCA Using prcomp in R

I'm trying to do principal component analysis (PCA) in R using the prcomp function. My input is a large matrix of 1,188 observations (rows) and 15,462 features (cols). I input this to the function ...
2
votes
0answers
50 views

Why linear transformation can improve classification accuracy when the dimensionality of data is high?

Let $X$ be an $m\times n$ ($m$: number of records, and $n$: number of attributes) dataset. When the number of attributes $n$ is large and the dataset $X$ is noisy, classification gets more ...
0
votes
1answer
115 views

Result from Step by step PCA implementation differs from `matplotlib.mlab.PCA()`, would be nice if someone can help me finding the source

I was reading this nice article and tried to implement this step by step guide in Python, and then I compared the results using the Python function from the matplotlib library: ...
0
votes
1answer
27 views

Finding most significant data set within data

Basically, I'm conducting research based on two types of data: Noise levels and the temperature of a room. I've recoded data for 2 days.. I am using the spearman's correlation methods to determine ...
0
votes
0answers
52 views

Dimension reduction for likert questions, cronbach alpha

I need some input on how to proceed with my data. I have collected data from household survey on 180 sample to find out the importance given to the attributes in a residential location decision. The ...
3
votes
1answer
160 views

Machine learning techniques for spam detection, and in general for text classification

I am going to configure a system for spam detection. What I have is a dataset of labeled (spam/not-spam) strings containing, mostly, sentences. I have a background in machine learning techniques, but ...
1
vote
0answers
60 views

Comparing isomap residual variance to pca variance

I am using R princomp function (from stats package) to run a PCA on a data set and I want to compare its output to that of the nonlinear dimensionality reduction method ISOMAP, which I am using under ...
1
vote
0answers
40 views

Define a hypersherical neighborhood around a target point with radius r'

This is a homework problem that I need some help with. QUESTION: We assume that N data points are uniformly distributed in a 100-dimensional unit hypersphere (i.e. $r = 1$) centered at the ...
1
vote
1answer
83 views

Linear model predictor selection. Which method to use ?

From what I understand, there are 3 main types of predictor selection method for linear models, namely, 1 Subset Selection, 2 Shrinkage and 3 Dimension Reduction. The subset selection includes the ...
0
votes
1answer
42 views

Combining variables

I'd like to combine several variables into one variable. Here is some context: Let's say I have two variables Red.Beads and ...
0
votes
1answer
38 views

When accuracy in the cross-validation process less,is reducing the features a good idea?

I am doing a project for classifying the presence of cars/bikes in an image.I have extracted the features from the images(data-set of cars and images not belonging to that of cars) and applied K-means ...
2
votes
3answers
134 views

Performing PCA with only a distance matrix

I want to cluster a massive dataset for which I have only the pairwise distances. I implemented a k-medoids algorithm, but it's taking too long to run so I would like to start by reducing the ...
2
votes
0answers
26 views

Creating consensus from multiple methods of measuring the same entity with some missing values

Imagine we have C cars and D drivers, and each driver takes a large subset of these C cars in order to test the rate of fuel consumption for some fixed amount of fuel (let's assume that the number of ...
0
votes
0answers
22 views

two inversly correlated indepent variables…how to combine?

I have a few instances where two independent variables are almost perfectly correlated inversely usually in such an instance i would remove one in this instance the two variables each comprise: ...
1
vote
0answers
27 views

Multidimensional scaling of variables with multiple sub-features?

Let's say I have a year's worth of magazine issues (January, February, March, etc), and I want to visualize the differences among them. The classic example of multidimensional scaling (MDS) would have ...
2
votes
1answer
105 views

dimension reduction of discrete numerical data

I have a bivariate discrete numerical dataset and would like to reduce its dimensions to a single variable. A 9 x 8 table of counts of the (x,y) data values is: ...
1
vote
0answers
38 views

Fitting a parameterized 2D distribution

I frequently deal with datasets that rely on a binary classification based on a Pulse-Shape-Distribution (PSD) discrimination value. This is based on a 2d fit of the PSD value vs the amplitude of the ...