Questions tagged [sparse]

A sparse matrix is a matrix where many of the elements are zeros. The tag can also be used for sparsity in other contexts, such as regression models with sparsity, or the "bet on sparsity"-principle.

109 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
7
votes
0answers
815 views

Feature scaling for non-negative sparse data

Imagine you have many observations on which you want to run a classification algorithm. Each observation is characterized by a matrix of non-negative values. For all observations 90-98% of the values ...
5
votes
0answers
908 views

Compute Shannon entropy between every row of a large, sparse matrix

I have a sparse, binary matrix of user (rows) and items (columns). Each element of this matrix is either 0 or 1: ...
4
votes
0answers
39 views

Probability distribution associated with nuclear norm?

The $\ell_1$ norm of model parameters is often added to loss functions because it induces sparsity in the solution of the overall cost function: $$ c(\theta) = \log L(x|\theta) + \lambda ||\theta||_1$$...
4
votes
0answers
348 views

How is explained variance in sparse PCA calculated?

Sparse PCA is a technique proposed by Zou et all in this paper. In usual PCA the obtained loadings are orthonormal, and the resulting scores are uncorrelated. However, in sparse PCA you give up these ...
4
votes
0answers
50 views

using sparse models to accelerate training

I was going through a tutorial that highlights the importance of using sparse models to get "lightning fast models" i.e. to accelerate the training process. What are the issues in using a sparse ...
4
votes
0answers
165 views

Sparse Representation, Sparse Learning, Sparse Coding, Group Sparse Coding and Group Sparse Learning?

I'm really confused with these terms for the relations and difference between them: Sparse Representation Sparse Learning Sparse Coding Group Sparse Coding Group Sparse Learning Sparse Dictionary ...
4
votes
0answers
818 views

Gibbs sampling for spike and slab priors

In Spike and slab variable selection (equation 4) there is a model setup of the form $\beta_k | \lambda_k, \tau_k \sim \text{Normal} (0, \lambda_k \tau_k^2)$ $\lambda_k | \nu_0, w \sim (1-w)\delta_{\...
3
votes
0answers
46 views

What is the appropriate metric for determining distance / dissimilarity of sparse, high dimensional data in PCA space?

I'm working with scRNA-seq data (~96% sparse, high dimensional), and am trying to determine distances between the cells in PCA space - NOT for the specific purpose of clustering. The principal ...
3
votes
1answer
199 views

Combining many sparse binary variables

Based on kjetil b halvorsen suggestion, I rephrased my problem: My problem is analogous to the following: i am supposed to predict if a high school student will go to university (Yes/No). I have ...
3
votes
0answers
30 views

Probability of sparse spectrum

Consider a vector $v$ such that $v \sim \mathrm{Unif}(\mathbb{S}^{d-1})$, the uniform distribution on the unit sphere in $d$ dimensions. Question: is there an upper bound on the probability that $v$ ...
3
votes
0answers
295 views

How to conduct a principal component analysis on data set with large number of zeros

I have data for percentage cover of plant species in 500 sites. There are columns for 30 different species in the data set and I would like to drastically reduce this down to a manageable number of ...
3
votes
1answer
405 views

Simulation of low rank and sparse matrix

I am having trouble simulating a matrix which is low rank and sparse (sparse along both rows and columns). One way to simulate a low-rank matrix is by generating a random matrix, then taking SVD and ...
3
votes
0answers
55 views

How to prove oracle properties in Fan and Li (2001) paper

I am studying Fan and Li's 2001 paper "Variable selection via nonconcave penalized Likelihood andits oracle properties" but I am having troubles understanding Theorem 1 proof (page 1359). I ...
3
votes
1answer
2k views

Optimization algorithms for sparse data

For couple of weeks now I've been dealing with a classification problem involving a sparse dataset. To be more specific, for each input $x^{(i)}$, knowing that I have 1000 features, I've only 5 to 10 ...
3
votes
0answers
96 views

Are factorization machines robust to outliers?

Factorization machines (FMs) seem great for modeling very sparse data. However, I have not come across much discussion regarding the impact of outliers. If FMs are robust, why is that so?
3
votes
0answers
1k views

Efficient/feasible sparse matrix inversion in R

I am looking to perform a 2-stage least-squares estimation with sparse matrices in R, in the style of Bramoulle et al (J. Econometrics 2009). Specifically, let: G be a very sparse block-diagonal ...
3
votes
0answers
3k views

Typical range of values for TFIDF

I am working on a text corpus. Each line contains between 10 and 50 words. There are around 25 000 words in the whole text and 1 000 000 lines. I turned this corpus into its tf-idf representation. I ...
3
votes
0answers
409 views

sparse covariance/correlation thresholding

In our project, we would like to do some optimization on sparse matrices. The idea is to scrape massive amounts of data, form a covariance/correlation matrix, and form a sparsity pattern basically by ...
3
votes
0answers
162 views

Random classification forests for extremely sparse response variables

I have a response variable that can be $A,B,C$. It is very sparse, meaning 99% of the sample is $B$ and the rest is approximately evenly divided between $A$ and $C$. How do I predict this variable in ...
3
votes
0answers
102 views

Sparsity regularization for eigenvectors

One way to think about finding the eigenvectors of a matrix $A$ is that they are the critical points of the functional $\vec x^\top A \vec x$ subject to $\|\vec x\|_2=1$. To regularize this problem, ...
3
votes
0answers
212 views

Is there a (parametric) distribution over sparse symmetric, binary matrix-valued random variables?

The title says it all, really. I'm aware of the Wishart distribution for symmetric, nonnegative-definite matrix-valued random variables, and am looking for something along these lines, but for sparse ...
2
votes
0answers
19 views

Clustering text embeddings: TF-IDF + BERT Sentence Embeddings

I am trying to cluster a few thousand forum posts that are similar in content to Stackoverfow. So far, I have tried two main approaches to represent the posts: TF-IDF Sentence embedding based on BERT....
2
votes
1answer
19 views

Scaling a sparse matrix

I want to apply sparse PCA to a sparse matrix. I was wondering if scaling to mean 0 and unit variance would be appropriate given that my input is sparse?
2
votes
1answer
91 views

Power simulation on glmer.nb gave strange results

I would like to ask for solution or advice on strange result that glmer.nb from lme4 generated when simulating using simR package. I’m working on longitudinal gut microbiome abundance data (23 ...
2
votes
0answers
34 views

How to interpret the learning curve of a LinearRegression for sparse data?

I have a dataset of shape ca.(4800, 350). Both the dataset X as well as the response y is very sparse (ca. 3500 samples with y=0). I wanted to take a look at the learning curve to estimate the bias-...
2
votes
0answers
173 views

Does anyone know the rank of the Netflix Prize dataset?

I'm looking into the Netflix Prize at the moment. We model the dataset as an $n \times m$ matrix, where $n$ is the number of users and $m$ is the number of movies. Does anyone know the rank of the ...
2
votes
0answers
34 views

Is low rank finite-iteration manifold identification possible?

In sparse optimization, I am trying to solve the problem $$ \min_{x\in \mathbb R^{n}} \quad f(x) + \|x\|_1 $$ and at optimality, $x^*$ may be sparse. If I define the sparse manifold as $\mathcal M = ...
2
votes
0answers
240 views

GAN and NN for sparse data

I have a set of images which represent some correlated sparse data $x_1,\ldots ,x_n$. there are a number of specific pixels in the images which might hold value or not (with some probability), while ...
2
votes
0answers
357 views

How to transform my sparse count data into normal distribution?

I am running glm on beetle counts data. My predictors are environmental variables and my response variable is the number of beetles. I ran three glms: The response variable $Y_1$ is the total number ...
2
votes
0answers
1k views

how to handle the sparse data in machine learning for novelty detection using SVM in r?

I have a data frame wherein 89000 records 60% of the values are missing I am replacing the missing values by ...
2
votes
0answers
79 views

How can lasso CCA be solved using LARS?

According to paper By Sun, Ji an Ye; A Least Squares Formulation for Canonical Correlation Analysis http://www.machinelearning.org/archive/icml2008/papers/270.pdf CCA can be reformulated as a least ...
2
votes
0answers
305 views

t-SNE on a small sparse matrix

I performed t-SNE on a this small sparse matrix with 2 identical points: ...
2
votes
0answers
86 views

Bayesian dictionary learning derivations

I am trying to do the derivations and implementation of dictionary learning/sparse coding in a Bayesian way. I am not sure if the derivations are correct, or maybe my approach is totally wrong. So ...
2
votes
2answers
468 views

Tuning parameter in the LASSO/group LASSO

I have a problem regarding the tuning parameter $\lambda$ in the LASSO or group LASSO. Suppose I want to find a matrix $\mathbf{A} = [\mathbf{a}_1,...,\mathbf{a}_n]\in\cal{C}^{m\times n}$ that ...
2
votes
0answers
80 views

Are there kernel-based one-class sparse kernel-based outlier detection methods, e.g. one-class Relevance Vector Machine?

I have a commercial outlier detection problem in moderate dimension (8-25). We have a limited number of true positive tags and can roughly evaluate performance of various methods. So far, the 1-...
2
votes
0answers
181 views

Clustering algorithm advice for extracting key features in sparse data

I have the following dataset: consider a dataset $X$ of $1400 \times 600$. The rows represent households at time $1 \leq t \leq 14$. So I have $100$ households. The columns represent the programs that ...
2
votes
0answers
583 views

Miss Forest & Iterative PCA : How to handle very sparse matrix imputation?

I am currently benchmarking matrix completion methods (k-NN, RandomForest and iterative PCA) on multivariate normal data in which I introduce a certain proportion of NA (5 to 95%). My performance ...
2
votes
0answers
588 views

How to go from sparse matrix to linear regression model (using SVD)?

I am trying to replicate the Kosinski, Stillwell, & Graepel (2013) study about predicting private traits and attributes from Facebook like data for study purposes. First I have admit, however, ...
2
votes
0answers
107 views

Instances of sparse covariance matrices

I am trying to find large datasets with inherently sparse covariance matrices, to be tested with our algorithm. Basically, we will take the sample covariance matrix and enforce some structured ...
2
votes
0answers
770 views

Reduction of sparse features for machine learning

I'm trying to use a 1D histogram as a feature for machine learning. A histogram instance can be very sparse and the range of its bins is theoretically unbounded. Moreover it is expected that non-zero ...
2
votes
0answers
835 views

The name data sparsity in different applications

I am recently surveying the techniques or algorithms which handle the data sparsity problems in various fields. And I find quite similar name "data sparsity" or "sparse data" is used including the ...
2
votes
0answers
338 views

Confusion related to convexity and non convexity of a problem

I was reading this paper related to sparse PCA enter link description here I didn't understand how they said that the function was convex or non convex I wanted to know how in the screenshot ...
2
votes
0answers
589 views

Sparse matrix representation of a spline interpolation

I use spline interpolation within a statistical model, and the transpose of the operator turns up in the gradient of the log-likelihood. Let me set up some notation first. If $x_1 \ldots x_n$ are a ...
1
vote
0answers
17 views

Enforcing sparsity constraints that make use of spatial contiguity

I have a deep learning network that outputs grayscale image reconstructions. In addition to good reconstruction performance (measured through mean squared error or some other measure like psnr), I ...
1
vote
0answers
22 views

Is K-medoids / partitioning around medoids (PAM) appropriate for clustering data with many zero values?

I need to cluster a matrix which contains zero values. I am clustering three separate sets of 24 values. The first two are non-zero and represent hourly ambient temperature (in K) and electrical ...
1
vote
0answers
36 views

For ridge regression, show if $K$ columns of $X$ are identical then we must have same corresponding parameters

Show if $K$ columns of $X$, $({X_{j1}, X_{j2}...X_{jk}}) $are identical then we must have $\hat\beta_{j1},\hat\beta_{j2},...\hat\beta_{jk} $ are same in the ridge regression: $$\hat\beta = \underset{...
1
vote
0answers
66 views

Sparse PCA for $p >> n$ solution with Elastic Net

I was reading about the sparse principal component approach by Zou, Hastie and Tibshirani but I do not quite understand how they handle the $p \gg n$ case in their paper. To derive the sparse axis, ...
1
vote
0answers
37 views

Is it useful to use sparse regression (e.g. Lasso) when the number of observations is significantly larger than the number of covariates?

I'm learning about penalized/sparse regression and I noticed that the examples used for penalized/sparse regression, e.g. Lasso, are usually cases where the number of observations is significantly ...
1
vote
1answer
101 views

finding sparse regions in time series data

I have several hundred years of church baptisms that will be searched by people wanting to find the baptisms of their ancestors. I want to call attention to periods in the records in which the number ...
1
vote
0answers
175 views

Normalizing sparse matrix by mean, should the mean be calculated excluding zero?

I have very sparse matrix (70% sparsity) which I want to normalize by mean. I tried using mean both include and exclude zero. The histogram between count (y-axis) and value (x-axis) shows The value ...