Questions tagged [sparse]

A sparse matrix is a matrix where many of the elements are zeros. The tag can also be used for sparsity in other contexts, such as regression models with sparsity, or the "bet on sparsity"-principle.

Filter by
Sorted by
Tagged with
0
votes
1answer
45 views

How to create bins for sparse data?

There is a column that has 10 rows of continuous data , 8 of these 10 rows has 0 and rest has 5 and 10. How can I divide these into 5 bins.(as rest of my columns which are significant has 5 bins).
3
votes
1answer
1k views

Feature selection for very sparse data

I have a dataset of dimension 3,000 x 24,000 (approximately) with 6 class label. But the data is very sparse. The number of non-zero values per sample ranges from 10-300 (approx) out of 24,000. The ...
2
votes
2answers
170 views

How can one generate a sequence of unique k-sparse matrices without rejection sampling for an arbitrary k efficiently?

I would like to efficiently generate $k$-sparse matrices, i.e., matrices that have only $k$ nonzero entries. The catch is that all such matrices must be different. Matrices are filled with 1 at the $k$...
1
vote
0answers
38 views

Periodicity in noisy data and the usefulness of differencing

Disclaimer: I don't have great stats knowledge. I'm doing some exploratory data analysis with the goal of detecting whether or not there is periodicity in the data set. I have a collection of photon ...
2
votes
2answers
1k views

Can I use PCA with mixed and sparse data types?

I am trying to reduce the dimensionality of a data set of about 100'000 rows and 1'000 columns, in order to cluster the individual observations with k-means. I tried PCA with rescaling (i.e., ...
1
vote
2answers
558 views

Calculating the log-determinant of large, sparse covariance matrices

As part of calculating the log-density of a very large multivariate normal distribution, I need to evaluate the following log-determinant: $$ f = \mathrm{ln}\left( \mathrm{det}(\mathbf{XAX}^\top + \...
1
vote
2answers
707 views

Criteria for choosing between PCA and sparse PCA

A bit of a neophyte question: I want to conduct data reduction on an NLP dataset 2000+ variables and 100000 plus cases. I am looking at different data reduction techniques discussed in "Robust ...
4
votes
1answer
2k views

How does glmnet handle larger datasets?

I'm looking to fit a model with about 1k-40k variables and up to a few million observations. Can anyone with a bit more experience speak to its performance for larger datasets? It looks like I can ...
0
votes
1answer
235 views

Is there a machine learning algorithm that can be trained with pairs of integer sets?

Like, the training set is composed of positive examples (s1, s2) where s1 is an integer set and s2 another integer set. s1 and s2 may have different cardinality. Negative examples are similar: pairs (...
1
vote
0answers
29 views

How to combine purchase and click data togehter in sparse matrix

my problem is the following: I have purchase probability estimations of different products. The model behind don't take care of any inter-correlations through these products. So my task is to re-...
3
votes
1answer
2k views

Dealing with Sparse Matrices and multiple numerical features when training algorithm

I have a data frame that looks as follows: ...
6
votes
2answers
1k views

When and why do we use sparse coding?

Sparse coding is described as "given an input $X$, finding a latent representation $h$ such that h is sparse and the input can be reconstructed as well as possible." (source: https://www.youtube.com/...
1
vote
2answers
192 views

How do I do a change point analysis on a sparse data set in python?

So, I have some data from video game playtests, where players were allowed to play a game at home for a week, and were asked to fill out a daily survey. In particular, they were asked to rate a ...
2
votes
1answer
794 views

CNN where pixels are constituted by large, potentially sparse vectors

I'd like to apply a CNN to a problem where the image is essentially a matrix representation of a geographical map where matrix indices correspond to the locations of buildings and roads. Each ...
4
votes
1answer
2k views

Basis pursuit denoising (BPDN) and LASSO with a given measurement error?

I am having some difficulties to understand the difference between: Basis Pursuit DeNoising (BPDN) which is often stated as: $min \left \| x \right \|_1 s.t \left \|Ax-b \right \|_2 \leq \...
0
votes
0answers
15 views

Marginal Homogeneity and (SPARSE) Ordered Categories

I'm trying to compare pre-test responses on a 5-Point Likert item to post-test responses on (the same) 5-Point Likert Item. The responses I observed as shown in the following table: \begin{array} {|...
3
votes
1answer
2k views

Optimization algorithms for sparse data

For couple of weeks now I've been dealing with a classification problem involving a sparse dataset. To be more specific, for each input $x^{(i)}$, knowing that I have 1000 features, I've only 5 to 10 ...
2
votes
0answers
82 views

How can lasso CCA be solved using LARS?

According to paper By Sun, Ji an Ye; A Least Squares Formulation for Canonical Correlation Analysis http://www.machinelearning.org/archive/icml2008/papers/270.pdf CCA can be reformulated as a least ...
2
votes
1answer
270 views

Visualization strategy for sparse binary data

I have ≈90 000 medical cases with ≈250 distinct diagnoses between them, distributed over 20 years; the most common of which applies to roughly half, and the least common applies to a handful. ETA: ...
2
votes
1answer
112 views

In regression, what if a factor is very sparse?

I am recently working on LASSO for my commercial car usage data. A problem I am facing now is that, I have around 200 factors, 100000 samples. However, quite a few of the factors are very sparse: only ...
1
vote
0answers
204 views

Sparse Weighted Local Least Squares

Introduction: Say I have a design matrix $X\in \mathbb{R}^{m\times n}$ with rows $x^{(i)} \in \mathbb{R}^{n}, i = 1,...,m$ and I also have a data vector $y \in \mathbb{R}^m$. I construct a "local" ...
1
vote
0answers
426 views

Sparse Pattern Recognition Using Convolutional Neural Networks

Context of Problem I am trying to troubleshoot communication issues between process sensors of a large manufacturing company. I have been given 1-year worth of data for 1,000 distinct processes (...
1
vote
0answers
664 views

Anomaly detection on one dimensional data

I'm looking for a general method to detect outliers in one dimensional data. That is to say without setting any anomaly threshold. Below is an example of my data : Red points are far enough from the ...
4
votes
1answer
4k views

Dealing with sparse categories in binary cross-entropy

In Keras, I'm using something similar to the Keras IMDB example to build a topic modelling example. However, unlike the example, which has a single "positive/negative" classification, I have over a ...
1
vote
0answers
346 views

Sparse SVD weird behavior

I'm finding the SVD for this matrix (in Python), and truncating it to the 4 first components: When using: ...
1
vote
0answers
474 views

Deep Autoencoder in TensorFlow to learn pseudo-sentence embeddings

I am stuck with a problem since days which is to find a right set-up for an Autoencoder that learns dense representations of sparse textual representations, word order is not relevant which is why I ...
1
vote
0answers
539 views

Pseudoinverse of large sparse matrix in R [closed]

I am trying to calculate the pseudoinverse of a large sparse matrix in R using the singular value decomposition. The matrix is roughly 240,000 x 240,000, and I have it stored as type ...
0
votes
0answers
102 views

What is the best way to determine whether a cluster is dense or sparse?

I tried using sum of distance from the center to every point. It did not perform well. Basically if I was given a part of either of these clusters, using this measure I want to be able to determine to ...
-1
votes
1answer
207 views

Kmeans for a data matrix containing both dense and sparse columns?

Assume the matrix contains one dense column, which consists of continuous values between 1-100. The other columns are binary values and are sparse. When applying Kmeans to such as matrix, does the ...
13
votes
4answers
603 views

$L_p$ Norms - What is special about $p=2$?

An $L_1$ norm is unique (at least partly) because $p=1$ is at the boundary between non-convex and convex. An $L_1$ norm is the 'most sparse' convex norm (right?). I understand that the $p=2$ ...
24
votes
1answer
28k views

Difference between missing data and sparse data in machine learning algorithms

What are main differences between sparse data and missing data? And how does it influences machine learning? More specifically, what effect sparse data and missing data have on classification ...
2
votes
1answer
50 views

VAR model: combining multiple short samples generated by similar DGPs

I have a problem which only has 35 observations. I like to apply around 6 variables in a VAR model to predict 25 observations into the future. Clearly, the data set is way too small. However, I do ...
3
votes
0answers
458 views

Poor recurrent neural network performance on sequential data [duplicate]

I have a dataset of energy measurements taken every minute from the energy footprint of home appliances. Based on that I am trying to detect human presence in the house. Since the data is sequential, ...
0
votes
1answer
180 views

Minimization of expected risk

On page 9 of "High Dimensional Sparse Econometric Models: An Introduction (2011)," Belloni and Chernozhukov explain in Remark 1 that the expected risk of a sparse estimator is $$\min_{\beta\in\...
2
votes
0answers
311 views

t-SNE on a small sparse matrix

I performed t-SNE on a this small sparse matrix with 2 identical points: ...
8
votes
1answer
2k views

Compressed Sensing relationship to L1 Regularization

I understand that compressed sensing finds the sparsest solution to $$y = Ax$$ where $x \in \mathbb{R}^D$, $A \in \mathbb{R}^{k \times D}$, and $y \in \mathbb{R}^{k}$, $k << D$. In this way we ...
1
vote
0answers
678 views

Deep neural network: categorical cross entropy with l1-norm (sparsity)

I am using a deep neural network in order which consists of: 1i nput layer, 2 hidden (dense) layers, 2 dropout layers (right after each dense layer), 1 softmax classifier (output). The cost function ...
3
votes
2answers
662 views

Large-scale MAE regression in R

I have a large, sparse dgCMatrix matrix in R: ~200,000 rows ~150,000 columns ~1,000,000,000 non-zero entries R code to generate the matrix: ...
2
votes
0answers
87 views

Bayesian dictionary learning derivations

I am trying to do the derivations and implementation of dictionary learning/sparse coding in a Bayesian way. I am not sure if the derivations are correct, or maybe my approach is totally wrong. So ...
0
votes
1answer
142 views

Is correlation, as a metric in clustering, affected by sparsity? [closed]

I want to correlate one sample to a set of classes' centroids (i.e. for each class, the sample composed by the median value of each feature in the set of samples of the class) to understand which ...
4
votes
0answers
166 views

Sparse Representation, Sparse Learning, Sparse Coding, Group Sparse Coding and Group Sparse Learning?

I'm really confused with these terms for the relations and difference between them: Sparse Representation Sparse Learning Sparse Coding Group Sparse Coding Group Sparse Learning Sparse Dictionary ...
1
vote
0answers
228 views

Is there a statistical test for sparsity?

We are developing an algorithm that tries to reconstruct/impute missing data from sparse datasets. I would like to know how to asses or quantify the sparsity of a dataset? Is there an appropriate ...
2
votes
2answers
500 views

Tuning parameter in the LASSO/group LASSO

I have a problem regarding the tuning parameter $\lambda$ in the LASSO or group LASSO. Suppose I want to find a matrix $\mathbf{A} = [\mathbf{a}_1,...,\mathbf{a}_n]\in\cal{C}^{m\times n}$ that ...
3
votes
0answers
98 views

Are factorization machines robust to outliers?

Factorization machines (FMs) seem great for modeling very sparse data. However, I have not come across much discussion regarding the impact of outliers. If FMs are robust, why is that so?
0
votes
0answers
135 views

Background required for understanding Robust PCA and Low-Rank Sparse Decomposition

My current knowledge is Linear Algebra, basics of statistics and Machine Learning (Andrew Ng's ML Coursera). I have a very good understanding of classic PCA and I know how to implement it in python ...
1
vote
0answers
79 views

e-SVM performance vs number of feature

I apply epsilon Support Vector Machine (e-SVM) to a regression problem via Weka. I have about 6000 features and 2000 samples. I order the feature respect to minimal-redundancy-maximal-relevance ...
5
votes
1answer
233 views

Any penalized ensemble classifiers in `Scikit-learn` that result in sparse solutions?

Is there an ensemble classifier that results in sparse solutions for the feature vector like Lasso Regression? With Logistic Regression, I can choose L1 penalization from the ...
4
votes
0answers
842 views

Gibbs sampling for spike and slab priors

In Spike and slab variable selection (equation 4) there is a model setup of the form $\beta_k | \lambda_k, \tau_k \sim \text{Normal} (0, \lambda_k \tau_k^2)$ $\lambda_k | \nu_0, w \sim (1-w)\delta_{\...
3
votes
0answers
1k views

Efficient/feasible sparse matrix inversion in R

I am looking to perform a 2-stage least-squares estimation with sparse matrices in R, in the style of Bramoulle et al (J. Econometrics 2009). Specifically, let: G be a very sparse block-diagonal ...
1
vote
1answer
958 views

How to find lasso beta estimates

I'm following this paper https://arxiv.org/pdf/1304.4773.pdf And for the moment I'm just trying to go through the steps for equation $(1.2)$ $$ \hat \beta_{lasso} = argmin_{\beta} \sum_{i}^{n} (y_{i}...