Questions tagged [sparse]

A sparse matrix is a matrix where many of the elements are zeros. The tag can also be used for sparsity in other contexts, such as regression models with sparsity, or the "bet on sparsity"-principle.

Filter by
Sorted by
Tagged with
2
votes
1answer
215 views

What is the method in dictionary learning that does not have an overcomplete dictionary?

What is the method in dictionary learning that does not have an overcomplete dictionary? And what is the difference in minimization between these two methods (one using overcomplete dictionary and ...
3
votes
1answer
264 views

Combining many sparse binary variables

Based on kjetil b halvorsen suggestion, I rephrased my problem: My problem is analogous to the following: i am supposed to predict if a high school student will go to university (Yes/No). I have ...
0
votes
0answers
35 views

LARS package in R with specific lambda value

I am trying to use the LARS package in R to obtain a Lasso estimate of the sparse coefficient vector, say $\hat{\beta}_{\text{sparse}}$, as opposed to a coefficient path. In other words, I am not ...
2
votes
2answers
500 views

Tuning parameter in the LASSO/group LASSO

I have a problem regarding the tuning parameter $\lambda$ in the LASSO or group LASSO. Suppose I want to find a matrix $\mathbf{A} = [\mathbf{a}_1,...,\mathbf{a}_n]\in\cal{C}^{m\times n}$ that ...
0
votes
0answers
35 views

Score Function for Sparse Mean Gaussian?

The Sparse Mean Gaussian model can be described by $ X \sim \mathcal{N}(\theta, \sigma^2 I_d)$ where $ \vert\vert\theta\vert\vert_{0} = s$ where $d$ is the dimension of the random variable, and $s$ ...
1
vote
1answer
236 views

Using an odds ratio when data is sparse

Suppose I have around 20 exposures that potentially affect an outcome and I want to see which exposures have bigger impacts on the outcome. So I want to calculate each exposures' odds ratios by ...
3
votes
1answer
2k views

Optimization algorithms for sparse data

For couple of weeks now I've been dealing with a classification problem involving a sparse dataset. To be more specific, for each input $x^{(i)}$, knowing that I have 1000 features, I've only 5 to 10 ...
0
votes
0answers
5 views

Is spare encoding a special case of spare autoencoding with ignoring non linear activation

we know that Sparse encoding is to minimize the objective function: $$\sum\limits_{n=1}^N\Big(||x_n - Az_n||^2 + \lambda\rho(z_n)\Big).$$ Here $A = [a_1,\cdots,a_M]...
0
votes
0answers
50 views

Spatial modelling of sparse count data

I am trying to model claim counts over for a given region. The data is very sparse. I am using the BESAG model from R INLA package. I am having a tough time to model the data. I am able to reproduce ...
3
votes
1answer
473 views

Simulation of low rank and sparse matrix

I am having trouble simulating a matrix which is low rank and sparse (sparse along both rows and columns). One way to simulate a low-rank matrix is by generating a random matrix, then taking SVD and ...
1
vote
0answers
26 views

outlier detection for sparse data in categorical variable

I have a big dataset with a column "clientid" and a categorical column "choice". I want to find out what are the clients that have strange combinations of choices (less frequent ...
0
votes
0answers
22 views

Calculate Data Sparsity - There are no 'zeros'

I am being asked to provide the percentage of data sparsity in my dataset. The challenge I am running into is that there isn't traditional sparsity in my data, no 'zeros', 'NAs', '-' etc.. I am ...
1
vote
1answer
73 views

How to deal with infrequent features in a linear regression model?

I am working on a linear model problem, $y =f(X)$ where $X$ has around 200 columns and around 300K rows not surprisingly, I am using LASSO to bring down the complexity of the model. but when I put the ...
2
votes
0answers
33 views

Clustering text embeddings: TF-IDF + BERT Sentence Embeddings

I am trying to cluster a few thousand forum posts that are similar in content to Stackoverfow. So far, I have tried two main approaches to represent the posts: TF-IDF Sentence embedding based on BERT....
0
votes
0answers
10 views

Difference between Structural Topic Modeling(STM) and SAGE (Sparse Additive Generative Model)?

I have read that STM combines 3 models of: (1) correlated topic model (CTM) (2) Dirichlet-Multinomial Regression (DMR) topic model (3) Sparse Additive Generative Model (SAGE) Is it correct to just ...
0
votes
0answers
12 views

Machine Learning algorithms for sparse database

I have a very sparse database with more than 200000 rows (instances) and 500 columns that lead to almost 100 million entries. However, only 205000 of the data are non zero, that is almost 0.2% of the ...
0
votes
0answers
7 views

Comparing sparse vectors

I am looking for a metric for comparing gene count tables. These are long columns of data (a few millions genes by a few dozen samples), with all non-negative entries, about 90% of which are zeros. ...
0
votes
1answer
119 views

How to do if the most training data is sparse

Consider a problem like this You have a customer profiling application(say classic telecoms data) You have customer transactions(lots of them) you want to find rules There is a data element which is ...
0
votes
0answers
25 views

Why can the L1 Norm be expressed as a constrain?

I am learning why the L1 regulariser (Lasso) is used to encourage sparsity in ML models. When describing the proof, I am seeing that the regularised minimisation cost function; $$ min(RSS(w) + \lambda*...
2
votes
1answer
53 views

Scaling a sparse matrix

I want to apply sparse PCA to a sparse matrix. I was wondering if scaling to mean 0 and unit variance would be appropriate given that my input is sparse?
7
votes
1answer
10k views

User segmentation by clustering with sparse data

Imagine that I have 100k users and 1k categories. For each user, up to 5 categories, I know how much money they have spent. Obviously my data is very sparse. Now I want to group users by the money ...
4
votes
0answers
48 views

Probability distribution associated with nuclear norm?

The $\ell_1$ norm of model parameters is often added to loss functions because it induces sparsity in the solution of the overall cost function: $$ c(\theta) = \log L(x|\theta) + \lambda ||\theta||_1$$...
8
votes
5answers
10k views

Cosine similarity on sparse matrix

I'm trying to implement item based filtering, with a large feature space representing consumers who bought (1) or did not buy (0) a particular product. I have a long tail distribution, so the matrix ...
2
votes
0answers
46 views

Are Spiking Neural Networks The Next Big Thing? [closed]

Intel recently announced their Loihi chip as part of their "Neuromorphic Computing" research, which is optimized for spiking neural networks (SNNs). I found an example of a problem in which ...
0
votes
0answers
45 views

NLP multiclass classification with many sparse classes

I am attempting to use natural language processing to geocode "addresses". The address is the result of a write-in of a survey where the respondent is instructed to give their city, state, ...
0
votes
0answers
37 views

Reconstruction metric robust to scaling, sparsity, and outliers?

I seek a reconstruction error metric with following properties: Robustness to sparsity: error decreases in presence of many zeros or small values (if predicted correctly) Scale invariance: error ...
1
vote
1answer
125 views

unsupervised anomaly detection on sparse data

Given that I have a very sparse data matrix with continuous features, like this dataframe for example ...
0
votes
0answers
25 views

Enforcing sparsity constraints that make use of spatial contiguity

I have a deep learning network that outputs grayscale image reconstructions. In addition to good reconstruction performance (measured through mean squared error or some other measure like psnr), I ...
3
votes
0answers
72 views

What is the appropriate metric for determining distance / dissimilarity of sparse, high dimensional data in PCA space?

I'm working with scRNA-seq data (~96% sparse, high dimensional), and am trying to determine distances between the cells in PCA space - NOT for the specific purpose of clustering. The principal ...
2
votes
2answers
34 views

L1 feature selection followed by exhaustive search

I'm working with a group on an ML project and one of the team members has proposed using L1 to reduce the feature space and then apply an exhaustive search with the reduced feature set. In each step, ...
84
votes
7answers
33k views

Euclidean distance is usually not good for sparse data (and more general case)?

I have seen somewhere that classical distances (like Euclidean distance) become weakly discriminant when we have multidimensional and sparse data. Why? Do you have an example of two sparse data ...
2
votes
2answers
199 views

Spectral norm of a sparse Gaussian matrix

Suppose $G$ is an $m \times n$ matrix such that each entry of $G$ is a standard normal variable. We know that the spectral norm of $G$ scales as $\sqrt m + \sqrt n$. Now, given a set of indices $S$ ...
0
votes
0answers
11 views

Custom Tensorflow v2.x Optimizer with Sparse update support

I am trying to contribute to tensorflow v2. I am done with _resource_apply_dense but i am struggling with _resource_apply_sparse. There are multiple ways to handle but there is no proper discussion ...
1
vote
1answer
118 views

Estimation of Sparse Panel Data

There are 1000 students and 100 teachers. Each teacher is given the answer scripts of randomly selected 100 students. So in total 10,000 answer scripts are judged. Now this is sort of panel data, but ...
0
votes
1answer
1k views

What's the best (Google chart) visualisation for displaying sparse timeline data across thousands of “columns”

I am trying to visualise a sparse dataset but am finding it hard to fit it into the standard categories of charts. I'm a developer building with Google Charts and I really want to stick with that ...
1
vote
1answer
131 views

What is a Dirac distribution on a hyperplane?

I'm trying to understand message passing for compressed sensing. I came acrross this distribution: As the title suggests, how does this distribution look like? I know the first products term in the ...
1
vote
2answers
192 views

How do I do a change point analysis on a sparse data set in python?

So, I have some data from video game playtests, where players were allowed to play a game at home for a week, and were asked to fill out a daily survey. In particular, they were asked to rate a ...
5
votes
1answer
243 views

Why over-complete sparse basis?

In This Stanford Tutorial, it says "Sparse coding is a class of unsupervised methods for learning sets of over-complete bases to represent data efficiently... While techniques such as Principal ...
0
votes
0answers
14 views

Sparsity-inducing priors for non-negative random variables

Which priors could be used for inducing sparsity on a random variable with non-negativity constraints?
9
votes
2answers
6k views

Can it be over fitting when validation loss and validation accuracy is both increasing?

Training a simple neural network over a very sparse matrix (Has 2400 features and 18000 train rows) for a binary classification problem. At the end of 1st epoch validation loss started to increase, ...
1
vote
0answers
30 views

Is K-medoids / partitioning around medoids (PAM) appropriate for clustering data with many zero values?

I need to cluster a matrix which contains zero values. I am clustering three separate sets of 24 values. The first two are non-zero and represent hourly ambient temperature (in K) and electrical ...
1
vote
1answer
112 views

How to train a Neural Network on sparse data?

I am trying to train a sequence model to extract specific substrings. I am working on extremely sparse text data (Sparsity ~ 0.03%, <1000 examples). After training for 500 epochs, the performance ...
2
votes
1answer
120 views

Power simulation on glmer.nb gave strange results

I would like to ask for solution or advice on strange result that glmer.nb from lme4 generated when simulating using simR package. I’m working on longitudinal gut microbiome abundance data (23 ...
0
votes
1answer
63 views

Using prior knowledge about correlated variable in ridge regression

I am wondering what methods are available for incorporating prior knowledge of some variable that is correlated with the unknown regression coefficients in a ridge regression. I have a sparse matrix ...
0
votes
1answer
34 views

Compact encoding (vectorization) of unbounded sets

Question I have a set of sets. Each set is unbounded. I would like to find a methodology to encode (vectorize) each subset. I am more specifically interested in memory efficient solutions. ...
2
votes
1answer
71 views

Why is computation of scores in Sparse PCA different from T=XP?

I am recently learning Sparse PCA. From a lately published paper All sparse PCA models are wrong, but some are useful. Part I: Computation of scores, residuals and explained variance I learned that ...
1
vote
0answers
43 views

For ridge regression, show if $K$ columns of $X$ are identical then we must have same corresponding parameters

Show if $K$ columns of $X$, $({X_{j1}, X_{j2}...X_{jk}}) $are identical then we must have $\hat\beta_{j1},\hat\beta_{j2},...\hat\beta_{jk} $ are same in the ridge regression: $$\hat\beta = \underset{...
1
vote
1answer
43 views

Why atoms in the dictionary of Dictionary Learning method are not required to be orthogonal?

According to Sparse Dictionary Learning (wiki), Sparse dictionary learning is a representation learning method which aims at finding a sparse representation of the input data (also known as sparse ...
27
votes
3answers
13k views

How exactly is sparse PCA better than PCA?

I learnt about PCA a few lectures ago in class and by digging more about this fascinating concept, I got to know about sparse PCA. I wanted to ask, if I'm not wrong this is what sparse PCA is: In PCA,...
2
votes
1answer
794 views

CNN where pixels are constituted by large, potentially sparse vectors

I'd like to apply a CNN to a problem where the image is essentially a matrix representation of a geographical map where matrix indices correspond to the locations of buildings and roads. Each ...

1
2 3 4 5 6