Questions tagged [sparse]

A sparse matrix is a matrix where many of the elements are zeros. The tag can also be used for sparsity in other contexts, such as regression models with sparsity, or the "bet on sparsity"-principle.

Filter by
Sorted by
Tagged with
84
votes
7answers
33k views

Euclidean distance is usually not good for sparse data (and more general case)?

I have seen somewhere that classical distances (like Euclidean distance) become weakly discriminant when we have multidimensional and sparse data. Why? Do you have an example of two sparse data ...
8
votes
2answers
5k views

Confused by MATLAB's implementation of ridge

I have two different implementations of ridge in MATLAB. One is simply $\mathbf x = (\mathbf{A}'\mathbf{A}+\mathbf{I}\lambda)^{-1}\mathbf{A}'\mathbf b$ (as seen ...
3
votes
1answer
612 views

Amount and sparsity of data for recommender systems

I'm starting to work in a project that will have a recommender system as one of its components. I'm trying to figure out if I have the right type of data for the recommender. The data contains ...
4
votes
1answer
2k views

Basis pursuit denoising (BPDN) and LASSO with a given measurement error?

I am having some difficulties to understand the difference between: Basis Pursuit DeNoising (BPDN) which is often stated as: $min \left \| x \right \|_1 s.t \left \|Ax-b \right \|_2 \leq \...
27
votes
3answers
13k views

How exactly is sparse PCA better than PCA?

I learnt about PCA a few lectures ago in class and by digging more about this fascinating concept, I got to know about sparse PCA. I wanted to ask, if I'm not wrong this is what sparse PCA is: In PCA,...
23
votes
4answers
12k views

Is there a Random Forest implementation that works well with very sparse data?

Is there an R random forest implementation that works well with very sparse data? I have thousands or millions of boolean input variables, but only hundreds or so will be TRUE for any given example. ...
11
votes
2answers
16k views

difference between convex and concave functions

what is the difference between convex, non-convex, concave and non-concave functions? how will we come to know that the given function is convex or non-convex? and if a function is non-convex then it ...
3
votes
2answers
6k views

How to build a predictive model with a billion of sparse features?

I am making a model to learn a dataset which has a big feature number and sparse samples (I am planning to use logistic regression). The feature number can be as big as 1,000,000,000. It is sparse ...
2
votes
3answers
7k views

Automatic outlier detection in R

Our model processes millions of multivariate observations; manual outlier detection is impractical. I am looking for a method of automatic outlier detection. I have been trying to use R package <...
8
votes
3answers
582 views

Goodness of fit test on sparse contigency tables with high dimensionality

I have a vector of size 1x3500, which can be viewed as the 'known distribution'. It is simply a table of counts across 3500 groups (i.e. a contingency table). I also have $N$ other vectors of the same ...
7
votes
0answers
844 views

Feature scaling for non-negative sparse data

Imagine you have many observations on which you want to run a classification algorithm. Each observation is characterized by a matrix of non-negative values. For all observations 90-98% of the values ...
7
votes
1answer
173 views

Literature on $\ell_q$ LASSO, $q < 1$

I am not sure how is $\ell_q$-LASSO called, but here I am talking about LASSO regression, with $\| \beta \|_{\ell_q}$ regularization, $q< 1$. In popular literature, such as Elements of Statistical ...
12
votes
1answer
5k views

Does a sparse training set adversely affect an SVM?

I'm trying to classify messages into different categories using an SVM. I've compiled a list of desirable words/symbols from the training set. For each vector, which represents a message, I set the ...
9
votes
2answers
1k views

Selecting the number of sparse principal components to include in regression

Does anyone have experience with approaches for selecting the number of sparse principal components to include in a regression model?
3
votes
2answers
1k views

Shrinkage priors

I am building a Bayesian model where I to put shrinkage priors such as spike and slab, horseshoe prior, etc on some parameters for feature selection, but I am not able to decide which one is the best. ...
3
votes
1answer
7k views

what is sparse solution?

This graph is about panelty least square method, and it is said that once a parameter hits zero, it remains zero for larger garmma, and this results in a sparse solution. I am wondering what it means ...
0
votes
2answers
2k views

Weather data in time series predictions

Disclaimer: I know this is a long-ish post but I don't need code solutions just high level general direction approaches that are usually used in situations like these. So let's say I want to predict ...
4
votes
0answers
3k views

Typical range of values for TFIDF

I am working on a text corpus. Each line contains between 10 and 50 words. There are around 25 000 words in the whole text and 1 000 000 lines. I turned this corpus into its tf-idf representation. I ...
3
votes
2answers
3k views

Algorithm and R implementation of sparse PCA

Does anyone know where I can find an algorithm, as well as an R implementation of it, to carry out sparse principal component analysis (PCA)?
3
votes
1answer
2k views

Dealing with Sparse Matrices and multiple numerical features when training algorithm

I have a data frame that looks as follows: ...
0
votes
0answers
37 views

Reconstruction metric robust to scaling, sparsity, and outliers?

I seek a reconstruction error metric with following properties: Robustness to sparsity: error decreases in presence of many zeros or small values (if predicted correctly) Scale invariance: error ...
0
votes
1answer
493 views

Does Matching Pursuit and Soft Thresholding return the same minimizer?

I wanted to understand if the solutions (minimizers) obtained by Matching Pursuit algorithms (say Basis Pursuit denoising) and Soft Thresholding yielded the same minimizer (same solution or same ...