# Questions tagged [sparse]

A sparse matrix is a matrix where many of the elements are zeros. The tag can also be used for sparsity in other contexts, such as regression models with sparsity, or the "bet on sparsity"-principle.

302 questions
Filter by
Sorted by
Tagged with
8 views

### Similarity measure for sparse, ordered, binary vectors, with more weighting to True values

I have two sparse, ordered, binary vectors. The size of the vectors is around ~100. I am under the impression that cosine similarity is useful for sparse, ordered, binary vectors. For my purposes, it ...
• 69
38 views

### Evaluating Lasso's Unique Solution and its consequences in applications?

I've grasped from a paper (https://www.stat.cmu.edu/%7Eryantibs/papers/lassounique.pdf) that Lasso may not yield a unique solution when the number of variables (p) exceeds the number of observations (...
• 301
34 views

### High dimensional regression with millions of covariates/features

as a matter of preamble, I am a machine learning researcher. I am interested if this community can point me to research and work showing settings that have performed regression where the number of ...
• 93
21 views

### Understanding softmax as an activation function, and sparsity in data and gradients

I’m working on a project that includes a probabilistic model that uses one hots, and also occasionally partially freezes weights or zeros gradients to specific regions of the weights. In some parts of ...
1 vote
53 views

### Community detection (graph clustering) vs. distance matrix clustering

I need to cluster objects. Each object is described by the set of features, each of which is either '0' or '1'. '1' means that object has this feature. '0' means that there is no information that ...
1 vote
72 views

### Implication for a perfect fit in OLS regression

If $\hat{\beta} = (X'X)^{-1}X'y$ with $X$ being an $n \times k$ matrix, then as I understand it, as long as $k \leq n$, $X'X$ is invertible (as long as all other OLS assumptions are ...
• 85
20 views

### Intercept term of logistic regression in ADMM algorithm

On page 66, the authors of article of ADMM says that the algorithm can be modified to obtain the intercept term easily in the sparse logistic regression model. Can someone explain this easy ...
• 313
414 views

### Why not directly brute force sparsify the OLS estimator instead of using Lasso?

I have a question about the Lasso estimator. I understand that it is particularly useful in high-dimensional settings due to its sparsity-inducing properties. For instance, if the design matrix is ...
• 145
1 vote
36 views

### Reproducing results from classic dropout paper [closed]

In the classic paper "Dropout: A Simple Way to Prevent Neural Networks from Overfitting", there is a figure comparing the features learned by a one-layer autoencoder trained on MNIST with ...
• 639
1 vote
49 views

### Difference between GNN and sparsely connected feedforward NN

What characteristics differentiate a classic GNN and a sparsely connected feedforward NN (basically a modified fully-connected NN), where the sparse connectivity is given by a user-defined sparsity ...
• 11
16 views

### Can I sample from a multivariate normal when I can only compute matrix-vector products?

I want to sample from a distribution $\mathcal{N}(0, \Sigma)$ where all I have is the ability to calculate $\Sigma v$ for all $v$. Is there any algorithm such that I can compute $Lu$ for any $u$ such ...
15 views

### VAR models: Effects of sparsity and magnitude disparities on VAR dynamics and possible solutions

In a VAR model involving two (or more) time series, if one series has sparse data with low counts, while the other series has lower sparsity and higher values, are there any statistical or technical ...
• 161
13 views

### Optimal method for predicting outcome from additive, correlated, and sparse features?

Suppose I have many vectors which can take on any of three values, 0, 1, 2. These vectors affect an outcome being predicted, Y. Vectors add together: a vector "A" of the value 2 has twice ...
• 171
1 vote
61 views

### Nonlinear Sparse PCA

Given data $x_1, \dots, x_n \in\mathbb{R}^d$, I am looking for a nonlinear dimensionality reduction technique $f: \mathbb{R}^d \rightarrow \mathbb{R}^q$ that only uses a limited number of dimensions ...
75 views

### Empirical basis functions

Preliminary Consider $n$ individuals each with observed data $Z_i, i = 1, \ldots, n$. For each individual $i$, the longitudinal predictor $Z_i = \{Z_i(t_{i1}), \ldots, Z_i(t_{i,R_i})\}$ is measured ...
• 741
248 views

### Python's acf and Matlab's xcorr apparently give different magnitude (but same pattern) answers for some data

I have an experiment with time series data (spike rates). A Python script calculating their autocorrelation with statsmodels.tsa.stattools.acf was apparently giving ...
• 273
1 vote
64 views

• 543
40 views

### Bayes prior in MAP estimation corresponding to $\ell^0$ penalization

I gather that in the context of penalized least squares, we can interpret a penalty term as corresponding to a prior $\pi(\beta)\propto \exp\{-\text{pen}\}.$ Is this also true for $\ell^0$ ...
• 543
651 views

### Bayesian priors associated with regularization penalties

I gather that adding a penalty term to (linear) least squares minimization typically corresponds with choosing some prior for Bayes estimation in the normal linear regression model. A couple questions ...
• 543
### LASSO with $L_p$ norms for $1 < p < 2$?
For the sparse linear regression problem, minimizing the LASSO objective $\| X \beta - Y \|_2^2 + \lambda \| \beta \|_1$ is known to recover the optimal data generating parameter $\beta^*$ with the ...