Questions tagged [sparse]

A sparse matrix is a matrix where many of the elements are zeros. The tag can also be used for sparsity in other contexts, such as regression models with sparsity, or the "bet on sparsity"-principle.

Filter by
Sorted by
Tagged with
8 votes
1 answer
403 views

Why not directly brute force sparsify the OLS estimator instead of using Lasso?

I have a question about the Lasso estimator. I understand that it is particularly useful in high-dimensional settings due to its sparsity-inducing properties. For instance, if the design matrix is ...
user405777's user avatar
1 vote
0 answers
34 views

Reproducing results from classic dropout paper [closed]

In the classic paper "Dropout: A Simple Way to Prevent Neural Networks from Overfitting", there is a figure comparing the features learned by a one-layer autoencoder trained on MNIST with ...
Ari Herman's user avatar
1 vote
1 answer
35 views

Difference between GNN and sparsely connected feedforward NN

What characteristics differentiate a classic GNN and a sparsely connected feedforward NN (basically a modified fully-connected NN), where the sparse connectivity is given by a user-defined sparsity ...
12213119's user avatar
0 votes
0 answers
16 views

Can I sample from a multivariate normal when I can only compute matrix-vector products?

I want to sample from a distribution $\mathcal{N}(0, \Sigma)$ where all I have is the ability to calculate $\Sigma v$ for all $v$. Is there any algorithm such that I can compute $Lu$ for any $u$ such ...
J. Zeitouni's user avatar
0 votes
0 answers
14 views

VAR models: Effects of sparsity and magnitude disparities on VAR dynamics and possible solutions

In a VAR model involving two (or more) time series, if one series has sparse data with low counts, while the other series has lower sparsity and higher values, are there any statistical or technical ...
kk68's user avatar
  • 161
0 votes
0 answers
12 views

Optimal method for predicting outcome from additive, correlated, and sparse features?

Suppose I have many vectors which can take on any of three values, 0, 1, 2. These vectors affect an outcome being predicted, Y. Vectors add together: a vector "A" of the value 2 has twice ...
BigMistake's user avatar
1 vote
0 answers
59 views

Nonlinear Sparse PCA

Given data $x_1, \dots, x_n \in\mathbb{R}^d$, I am looking for a nonlinear dimensionality reduction technique $f: \mathbb{R}^d \rightarrow \mathbb{R}^q$ that only uses a limited number of dimensions ...
Claudio Moneo's user avatar
0 votes
1 answer
68 views

Empirical basis functions

Preliminary Consider $n$ individuals each with observed data $ Z_i, i = 1, \ldots, n$. For each individual $i$, the longitudinal predictor $Z_i = \{Z_i(t_{i1}), \ldots, Z_i(t_{i,R_i})\}$ is measured ...
ADAM's user avatar
  • 689
2 votes
1 answer
180 views

Python's `acf` and Matlab's `xcorr` apparently give different magnitude (but same pattern) answers for some data

I have an experiment with time series data (spike rates). A Python script calculating their autocorrelation with statsmodels.tsa.stattools.acf was apparently giving ...
Katie's user avatar
  • 263
1 vote
0 answers
56 views

What are some dimension reduction techniques applicable to sparse covariance matrices?

Suppose that I have $n\gg 500000$ observations, and I specify $$\mathbf{y} \sim \text{Normal}(\mathbf{X}\boldsymbol{\beta},\sigma^2_y\boldsymbol{\Sigma}_y + \tau \mathbf{K}\mathbf{K}^T),$$ where $\...
Ron Snow's user avatar
  • 2,069
1 vote
2 answers
281 views

Python package for making a rank-deficient sparse matrix full rank

I am running a regression with a sparse rank-deficient matrix where many columns are correlated with others. At the moment, I remove all columns with a correlation over 0.8. The matrix has 12k columns ...
miguelmorin's user avatar
0 votes
0 answers
14 views

Strategies to find associations between subset of variables

My post is more about ideas than one specific approach. My problem is pretty simple, I have simulated two matrices of noisy data, basically following correlated negative binomial distributions, and my ...
Mangnier Loïc's user avatar
2 votes
1 answer
328 views

MAE to find tuning parameter for lasso logistic regression

I have a classification problem. The actual outcomes are binary (0 or 1), but I want to predict probabilities, rather than predicting simply 0 or 1. I also want something with feature selection, since ...
dwolfeu's user avatar
  • 620
0 votes
0 answers
62 views

What are Dense and Sparse Models?

I'm watching an explanation of the SHAP algorithm by its author S Lundberg. At 22:45 in the video, one of the audience members asks a question about the models behind his graphs. He explains that one ...
Connor's user avatar
  • 595
1 vote
1 answer
32 views

What are the modes of a dictionary / transform basis?

So, I'm reading Steven Brunton's book, "Data Driven Science & Engineering", and I'm trying to understand what he means by mode in this following excerpt: Most natural signals, such as ...
Nyquist-er's user avatar
2 votes
1 answer
275 views

Parameter choice rules for L1 regularization?

I am solving an L1 regularized least squares (LASSO) of the form: $$ \arg \min_{\boldsymbol{x}} \frac{1}{2} {\left\| A \boldsymbol{x} - \boldsymbol{y} \right\|}_{2}^{2} + \lambda {\left\| \boldsymbol{...
user376024's user avatar
0 votes
0 answers
345 views

Multilabel Classification: Accuracy is very low. Metric or Model, which is inadequate?

In my multilabel classifaction problem, which I approach similarly to what can be see in this post: How does Keras handle multilabel classification?, the resulting accuracy only increases from 2% to 5%...
Viktor's user avatar
  • 101
1 vote
0 answers
26 views

Sparse Matrix Resulting in Rank Deficient Correlation Matrix

Suppose I have a matrix with 34 binary predictors and a large number of observations such that there are always 5 ones per observation (i.e., 5 of the predictors are randomly set to 1 in each ...
user10234121's user avatar
2 votes
0 answers
27 views

Is the Sparsity Pattern (Number of Non Zero Elements) of the Lasso Solution Monotonic as a Function of $ \lambda$? [duplicate]

For solving: $$\hat\beta^\lambda = \arg\min_{\beta \in \mathbb{R}^p} \|y - X \beta\|_2^2 + \lambda \|\beta\|_1,$$ We know that the values of $ \beta $ might increase for some $ {\lambda}_{2} > {\...
Eric Johnson's user avatar
2 votes
0 answers
29 views

Recommendation for books about statistical data analysis with sparse data ( a lot of zeros)

Evident from the title, i am looking for books about the general statistical data analysis techniques such as hypothesis testing, etc but for large and sparse dataset.
4 votes
2 answers
333 views

Creating a random sparse precision matrix?

In my current project, I want to create a random sparse precision matrix $\boldsymbol{P}=\boldsymbol{\Sigma}^{-1}$ (the inverse of a covariance matrix $\boldsymbol{\Sigma}$). My current procedure ...
J.Galt's user avatar
  • 555
6 votes
2 answers
227 views

How do Sparsity Priors help for Identifiability?

Let's say we have a Factor Analysis model with a latent variable $\mathbf{z}_t \in \mathbb{R}^k$: $$x_t = A z_t + \epsilon_t, \qquad \epsilon_t \sim \mathcal{N}(0, \Sigma)$$ Let $A \in \mathbb{R}^{g\...
N8_Coder's user avatar
  • 189
1 vote
1 answer
129 views

Features differ between classes

Good evening everyone. Regarding the topic related to Sparse Clustering (for example K-Means). For example, in "Witten DM, Tibshirani R. A framework for feature selection in clustering" the ...
Alessandro Pio Budetti's user avatar
0 votes
0 answers
98 views

Does dataset sparsity affect rate of convergence of model solution?

I have a model with not too many ordinal data. The model performs at a 90% accuracy. I am thinking of adding 13 ordinal variables and transforming them using one hot encoder. The transformation will ...
ityr554's user avatar
1 vote
0 answers
51 views

How to implement simple VAE with sparse tensor in Tensorflow

thank you for reading. I have been attempting to train a simple VAE on very sparse 2D and 3D data. So far I have been training using dense tensors which - I think - is resulting in horrible training ...
Zephrom's user avatar
  • 11
1 vote
0 answers
70 views

Multivariate time-series with asyncronous data

I have quote observations for 9 FX rates, which I would like to analyze via multivariate dynamic linear model (e.g., Chapter 16 in West and Harrison Bayesian Forecasting and Dynamic Models (1997). In ...
MikeRand's user avatar
  • 422
1 vote
2 answers
128 views

Does 'sparse' mean different in sparse feature & sparse representation?

In Google developers site, the meaning of the word sparse seems to contradict between the following two definitions: sparse feature: Feature vector whose values are predominately zero or empty... ...
Augustine Charly's user avatar
4 votes
1 answer
381 views

Car rental time series forecasting

I have the time series of car rentals/demand in a location. The time axis is every hour for 3 months. The y-axis is number of car rentals/demand. I want to do prediction for future hours given this ...
Jose_Peeterson's user avatar
2 votes
1 answer
57 views

How to use sparse PCA loadings in a regression?

I'm using Dr Frank Harrell's code in RMS 2nd edition. He goes into sparse PCA. Does anyone know how to code a regression model after getting the sparse component grid? ...
A_cor's user avatar
  • 21
3 votes
0 answers
32 views

Fitting Sparsed Constrained regression with non-negative coefficients adding to 1

I see a similar problem in How do I fit a constrained regression in R so that coefficients total = 1? Specifically, my model is $Y_i= \pi_1 X_1+\pi_2 X_2 +...+ \pi_K X_K +\epsilon_i$ with $\pi_k \ge 0$...
Siddhartha R Dalal's user avatar
1 vote
0 answers
158 views

T-testing on sparse data

Problem: I have two stochastic processes, $S_1$ and $S_2$, that frequently are zero, but occasionally have positive values with unknown probabilities $q_1$ and $q_2$. e.g. $$ S_1 = \{0,0,0,0,0,21,0,0,...
D'Arcy's user avatar
  • 11
1 vote
1 answer
97 views

Why lasso cannot be arbitrarily applied?

Consider any log likelihood function $f(\theta|x)$ where $x$ is data. I can consider $f(\theta|x)+\lambda||\theta||_1$ where $||\theta||_1$ is the standard $L_1$ norm. It seems that I cannot apply ...
user45765's user avatar
  • 1,406
1 vote
2 answers
939 views

How to solve the problem of having sparse data that would become too small when aggregated?

I have a dataset that provides the count of cyber incidents since 2011 for different countries and different attack types, and I want to use this data in a machine learning model to predict future ...
Traveling Salesman's user avatar
2 votes
1 answer
518 views

Is it wrong to run a Random Forest on high-dimensional, sparse, and unbalanced data?

I am learning about random forests, and I have been testing using R. I have doubts about whether I am doing something wrong given that my data are: sparse, high-dimensional, and unbalanced. Trying to ...
Doon_Bogan's user avatar
1 vote
0 answers
53 views

Working with sparse data in numpy and sklearn [closed]

I have a time series dataset geenrated from some electrophysiological data. I have a frequncy dataset and the matrix is quite sparse but huge, like it contains 0.005 s time bins for 2000+ neurons ...
Angus Campbell's user avatar
0 votes
0 answers
50 views

bootstrapped l2,1 least square does not produce sparse solution

I am trying to model an autoregressive model with $\ell$-2,1 regularization, where $\|X\|_{2,1}=\sum_i|\sum_jX_{ij}|$: $$y_{t+1} = wx_{t} + \lambda_1\|w\|_{2,1}, y\in \mathbb{R}^{n_1}, x\in \mathbb{R}^...
rando's user avatar
  • 236
0 votes
1 answer
154 views

Variable selection with sparse data

I have a dataset with 141 observations and 8 corresponding variables and I mean to apply a GLM to this dataset. However, a lot of observations lack either one or multiple variable values. So if I want ...
dumei's user avatar
  • 11
1 vote
1 answer
33 views

How to compare row entries in a sparse table with lots of missing values?

I have a dataset with ~1000 laptops and performance results across ~100 different benchmarks. Using the benchmark results, I want to give each laptop a single composite performance score, and rank the ...
Rafael Sofi-zada's user avatar
4 votes
1 answer
645 views

Autoencoder doesn't learn 'sparse' input images

I am trying to train an autoencoder with PyTorch on 2D images containing 2D Gaussian densities such as this: The images are of size 100x100 (I feed them into the autoencoder as 1x10000 tensors). The ...
user149206's user avatar
1 vote
1 answer
556 views

What is the difference between network sparsification and model pruning

What is the difference between network sparsification and model pruning? I watched USENIX ATC '21 - Octo: INT8 Training with Loss-aware Compensation and Backward Quantization for Tiny (at 01:29sec) ...
Mas A's user avatar
  • 203
2 votes
0 answers
88 views

What are the references for different active set selection methods for sparse Gaussian processes?

I am comparing the different sparse Gaussian process approaches within the fitrgp function in Matlab, but I am struggling to find references for the different choices within the function for the ...
MitchRR's user avatar
  • 21
0 votes
0 answers
128 views

Oracle inequality for LASSO

In chapter 6 of Statistics for High Dimensional Data (Peter BühlmannSara van de Geer), they focus on the normal linear regression model with fixed regressors and begin by stating OLS achieves a risk $\...
Golden_Ratio's user avatar
2 votes
1 answer
39 views

Bayes prior in MAP estimation corresponding to $\ell^0$ penalization

I gather that in the context of penalized least squares, we can interpret a penalty term as corresponding to a prior $\pi(\beta)\propto \exp\{-\text{pen}\}.$ Is this also true for $\ell^0$ ...
Golden_Ratio's user avatar
4 votes
1 answer
540 views

Bayesian priors associated with regularization penalties

I gather that adding a penalty term to (linear) least squares minimization typically corresponds with choosing some prior for Bayes estimation in the normal linear regression model. A couple questions ...
Golden_Ratio's user avatar
0 votes
0 answers
28 views

LASSO with $L_p$ norms for $1 < p < 2$?

For the sparse linear regression problem, minimizing the LASSO objective $\| X \beta - Y \|_2^2 + \lambda \| \beta \|_1$ is known to recover the optimal data generating parameter $\beta^*$ with the ...
Television's user avatar
2 votes
1 answer
213 views

If there is a sparse solution then is the smallest l1 norm solution at least as sparse?

Consider the linear equation $Ax=b$ where $A$ is a matrix and $b$ and $x$ are vectors. Suppose there exists a vector $x_S$ that solves this equation ($Ax_S=b$) and $x_S$ has $k$ entries of value $0$ (...
SecretlyAnEconomist's user avatar
0 votes
0 answers
41 views

Multiple regression with sparse X

I would like to run the multiple regression: $$y = X \beta + \epsilon \,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,(1)$$ with: $n \sim 2 \times 10^6$ $p \sim 10^4$ $X$ sparse 70% of rows have 1 non-...
Cingonius Varro's user avatar
2 votes
1 answer
145 views

In a sparse reward problem, is it possible to remove reward shaping once the RL agent trains long enough to consistently reach the final reward?

I'm new to machine learning, and primarily looking at it from the perspective of it's applications to control theory. In the application found in this paper, a RL agent attempts to land a spacecraft ...
jgholder's user avatar
  • 143
0 votes
0 answers
150 views

LARS package in R with specific lambda value

I am trying to use the LARS package in R to obtain a Lasso estimate of the sparse coefficient vector, say $\hat{\beta}_{\text{sparse}}$, as opposed to a coefficient path. In other words, I am not ...
Carl's user avatar
  • 1,186
1 vote
0 answers
113 views

outlier detection for sparse data in categorical variable

I have a big dataset with a column "clientid" and a categorical column "choice". I want to find out what are the clients that have strange combinations of choices (less frequent ...
DataLover's user avatar

1
2 3 4 5 6