Questions tagged [sparse]
A sparse matrix is a matrix where many of the elements are zeros. The tag can also be used for sparsity in other contexts, such as regression models with sparsity, or the "bet on sparsity"-principle.
304 questions
1
vote
0
answers
32
views
Low sample size in one of the binary classes
I have a dataset with 86 observations, and I want to make inference based on 10 categorical covariates with 22 levels, which results in a model matrix with 12 columns including intercept.
The problem ...
- 332
0
votes
0
answers
7
views
Sparse adjacency matrix for CAR in brms?
I want to fit a model in the R package brms. My data consists of spatial polygons and I want to use conditional autoregression to account for autocorrelation. This is possible in brms but requires a ...
- 119
0
votes
0
answers
16
views
Similarity measure for sparse, ordered, binary vectors, with more weighting to True values
I have two sparse, ordered, binary vectors. The size of the vectors is around ~100.
I am under the impression that cosine similarity is useful for sparse, ordered, binary vectors. For my purposes, it ...
- 199
0
votes
0
answers
60
views
Evaluating Lasso's Unique Solution and its consequences in applications?
I've grasped from a paper (https://www.stat.cmu.edu/%7Eryantibs/papers/lassounique.pdf) that Lasso may not yield a unique solution when the number of variables (p) exceeds the number of observations (...
- 363
0
votes
1
answer
41
views
High dimensional regression with millions of covariates/features
as a matter of preamble, I am a machine learning researcher. I am interested if this community can point me to research and work showing settings that have performed regression where the number of ...
- 93
0
votes
0
answers
27
views
Understanding softmax as an activation function, and sparsity in data and gradients
I’m working on a project that includes a probabilistic model that uses one hots, and also occasionally partially freezes weights or zeros gradients to specific regions of the weights. In some parts of ...
- 1
1
vote
1
answer
67
views
Community detection (graph clustering) vs. distance matrix clustering
I need to cluster objects.
Each object is described by the set of features, each of which is either '0' or '1'.
'1' means that object has this feature.
'0' means that there is no information that ...
3
votes
1
answer
94
views
Implication for a perfect fit in OLS regression
If $ \hat{\beta} = (X'X)^{-1}X'y $ with $ X $ being an $ n \times k $ matrix, then as I understand it, as long as $ k \leq n $, $ X'X $ is invertible (as long as all other OLS assumptions are ...
- 145
0
votes
0
answers
28
views
Intercept term of logistic regression in ADMM algorithm
On page 66, the authors of article of ADMM says that the algorithm can be modified to obtain the intercept term easily in the sparse logistic regression model. Can someone explain this easy ...
- 313
9
votes
1
answer
425
views
Why not directly brute force sparsify the OLS estimator instead of using Lasso?
I have a question about the Lasso estimator. I understand that it is particularly useful in high-dimensional settings due to its sparsity-inducing properties.
For instance, if the design matrix is ...
- 347
1
vote
0
answers
37
views
Reproducing results from classic dropout paper [closed]
In the classic paper "Dropout: A Simple Way to Prevent Neural Networks from
Overfitting", there is a figure comparing the features learned by a one-layer autoencoder trained on MNIST with ...
- 639
1
vote
1
answer
59
views
Difference between GNN and sparsely connected feedforward NN
What characteristics differentiate a classic GNN and a sparsely connected feedforward NN (basically a modified fully-connected NN), where the sparse connectivity is given by a user-defined sparsity ...
- 11
0
votes
0
answers
20
views
Can I sample from a multivariate normal when I can only compute matrix-vector products?
I want to sample from a distribution $\mathcal{N}(0, \Sigma)$ where all I have is the ability to calculate $\Sigma v$ for all $v$.
Is there any algorithm such that I can compute $Lu$ for any $u$ such ...
- 53
0
votes
0
answers
19
views
VAR models: Effects of sparsity and magnitude disparities on VAR dynamics and possible solutions
In a VAR model involving two (or more) time series, if one series has sparse data with low counts, while the other series has lower sparsity and higher values, are there any statistical or technical ...
- 161
0
votes
0
answers
16
views
Optimal method for predicting outcome from additive, correlated, and sparse features?
Suppose I have many vectors which can take on any of three values, 0, 1, 2. These vectors affect an outcome being predicted, Y. Vectors add together: a vector "A" of the value 2 has twice ...
- 195
1
vote
0
answers
64
views
Nonlinear Sparse PCA
Given data $x_1, \dots, x_n \in\mathbb{R}^d$, I am looking for a nonlinear dimensionality reduction technique $f: \mathbb{R}^d \rightarrow \mathbb{R}^q$ that only uses a limited number of dimensions ...
- 318
0
votes
1
answer
87
views
Empirical basis functions
Preliminary
Consider $n$ individuals each with observed data $ Z_i, i = 1, \ldots, n$. For each individual $i$, the longitudinal predictor $Z_i = \{Z_i(t_{i1}), \ldots, Z_i(t_{i,R_i})\}$ is measured ...
- 751
3
votes
1
answer
310
views
Python's `acf` and Matlab's `xcorr` apparently give different magnitude (but same pattern) answers for some data
I have an experiment with time series data (spike rates).
A Python script calculating their autocorrelation with statsmodels.tsa.stattools.acf was apparently giving ...
- 283
1
vote
0
answers
78
views
What are some dimension reduction techniques applicable to sparse covariance matrices?
Suppose that I have $n\gg 500000$ observations, and I specify
$$\mathbf{y} \sim \text{Normal}(\mathbf{X}\boldsymbol{\beta},\sigma^2_y\boldsymbol{\Sigma}_y + \tau \mathbf{K}\mathbf{K}^T),$$ where $\...
- 2,073
1
vote
2
answers
473
views
Python package for making a rank-deficient sparse matrix full rank
I am running a regression with a sparse rank-deficient matrix where many columns are correlated with others. At the moment, I remove all columns with a correlation over 0.8. The matrix has 12k columns ...
- 167
2
votes
1
answer
415
views
MAE to find tuning parameter for lasso logistic regression
I have a classification problem. The actual outcomes are binary (0 or 1), but I want to predict probabilities, rather than predicting simply 0 or 1. I also want something with feature selection, since ...
- 620
1
vote
1
answer
37
views
What are the modes of a dictionary / transform basis?
So, I'm reading Steven Brunton's book, "Data Driven Science & Engineering", and I'm trying to understand what he means by mode in this following excerpt:
Most natural signals, such as ...
- 13
2
votes
1
answer
448
views
Parameter choice rules for L1 regularization?
I am solving an L1 regularized least squares (LASSO) of the form:
$$ \arg \min_{\boldsymbol{x}} \frac{1}{2} {\left\| A \boldsymbol{x} - \boldsymbol{y} \right\|}_{2}^{2} + \lambda {\left\| \boldsymbol{...
- 121
0
votes
0
answers
417
views
Multilabel Classification: Accuracy is very low. Metric or Model, which is inadequate?
In my multilabel classifaction problem, which I approach similarly to what can be see in this post: How does Keras handle multilabel classification?, the resulting accuracy only increases from 2% to 5%...
- 101
1
vote
0
answers
29
views
Sparse Matrix Resulting in Rank Deficient Correlation Matrix
Suppose I have a matrix with 34 binary predictors and a large number of observations such that there are always 5 ones per observation (i.e., 5 of the predictors are randomly set to 1 in each ...
- 11
2
votes
0
answers
27
views
Is the Sparsity Pattern (Number of Non Zero Elements) of the Lasso Solution Monotonic as a Function of $ \lambda$? [duplicate]
For solving:
$$\hat\beta^\lambda = \arg\min_{\beta \in \mathbb{R}^p} \|y - X \beta\|_2^2 + \lambda \|\beta\|_1,$$
We know that the values of $ \beta $ might increase for some $ {\lambda}_{2} > {\...
- 387
2
votes
0
answers
35
views
Recommendation for books about statistical data analysis with sparse data ( a lot of zeros)
Evident from the title, i am looking for books about the general statistical data analysis techniques such as hypothesis testing, etc but for large and sparse dataset.
5
votes
2
answers
489
views
Creating a random sparse precision matrix?
In my current project, I want to create a random sparse precision matrix $\boldsymbol{P}=\boldsymbol{\Sigma}^{-1}$ (the inverse of a covariance matrix $\boldsymbol{\Sigma}$). My current procedure ...
- 585
6
votes
2
answers
280
views
How do Sparsity Priors help for Identifiability?
Let's say we have a Factor Analysis model with a latent variable $\mathbf{z}_t \in \mathbb{R}^k$:
$$x_t = A z_t + \epsilon_t, \qquad \epsilon_t \sim \mathcal{N}(0, \Sigma)$$
Let $A \in \mathbb{R}^{g\...
- 219
1
vote
1
answer
170
views
Features differ between classes
Good evening everyone.
Regarding the topic related to Sparse Clustering (for example K-Means).
For example, in "Witten DM, Tibshirani R. A framework for feature selection in clustering" the ...
0
votes
0
answers
114
views
Does dataset sparsity affect rate of convergence of model solution?
I have a model with not too many ordinal data. The model performs at a 90% accuracy. I am thinking of adding 13 ordinal variables and transforming them using one hot encoder. The transformation will ...
- 1
1
vote
0
answers
55
views
How to implement simple VAE with sparse tensor in Tensorflow
thank you for reading.
I have been attempting to train a simple VAE on very sparse 2D and 3D data. So far I have been training using dense tensors which - I think - is resulting in horrible training ...
- 11
1
vote
0
answers
82
views
Multivariate time-series with asyncronous data
I have quote observations for 9 FX rates, which I would like to analyze via multivariate dynamic linear model (e.g., Chapter 16 in West and Harrison Bayesian Forecasting and Dynamic Models (1997).
In ...
- 432
1
vote
2
answers
142
views
Does 'sparse' mean different in sparse feature & sparse representation?
In Google developers site, the meaning of the word sparse seems to contradict between the following two definitions:
sparse feature:
Feature vector whose values are predominately zero or empty...
...
4
votes
1
answer
409
views
Car rental time series forecasting
I have the time series of car rentals/demand in a location. The time axis is every hour for 3 months. The y-axis is number of car rentals/demand.
I want to do prediction for future hours given this ...
- 294
2
votes
1
answer
59
views
How to use sparse PCA loadings in a regression?
I'm using Dr Frank Harrell's code in RMS 2nd edition. He goes into sparse PCA.
Does anyone know how to code a regression model after getting the sparse component grid?
...
- 21
3
votes
0
answers
35
views
Fitting Sparsed Constrained regression with non-negative coefficients adding to 1
I see a similar problem in How do I fit a constrained regression in R so that coefficients total = 1?
Specifically, my model is
$Y_i= \pi_1 X_1+\pi_2 X_2 +...+ \pi_K X_K +\epsilon_i$ with $\pi_k \ge 0$...
1
vote
0
answers
186
views
T-testing on sparse data
Problem: I have two stochastic processes, $S_1$ and $S_2$, that frequently are zero, but occasionally have positive values with unknown probabilities $q_1$ and $q_2$. e.g.
$$
S_1 = \{0,0,0,0,0,21,0,0,...
- 11
1
vote
1
answer
98
views
Why lasso cannot be arbitrarily applied?
Consider any log likelihood function $f(\theta|x)$ where $x$ is data. I can consider $f(\theta|x)+\lambda||\theta||_1$ where $||\theta||_1$ is the standard $L_1$ norm.
It seems that I cannot apply ...
- 1,465
1
vote
2
answers
1k
views
How to solve the problem of having sparse data that would become too small when aggregated?
I have a dataset that provides the count of cyber incidents since 2011 for different countries and different attack types, and I want to use this data in a machine learning model to predict future ...
2
votes
1
answer
708
views
Is it wrong to run a Random Forest on high-dimensional, sparse, and unbalanced data?
I am learning about random forests, and I have been testing using R. I have doubts about whether I am doing something wrong given that my data are: sparse, high-dimensional, and unbalanced.
Trying to ...
- 131
1
vote
0
answers
61
views
Working with sparse data in numpy and sklearn [closed]
I have a time series dataset geenrated from some electrophysiological data. I have a frequncy dataset and the matrix is quite sparse but huge, like it contains 0.005 s time bins for 2000+ neurons ...
- 468
0
votes
0
answers
50
views
bootstrapped l2,1 least square does not produce sparse solution
I am trying to model an autoregressive model with $\ell$-2,1 regularization, where $\|X\|_{2,1}=\sum_i|\sum_jX_{ij}|$:
$$y_{t+1} = wx_{t} + \lambda_1\|w\|_{2,1}, y\in \mathbb{R}^{n_1}, x\in \mathbb{R}^...
- 328
0
votes
1
answer
166
views
Variable selection with sparse data
I have a dataset with 141 observations and 8 corresponding variables and I mean to apply a GLM to this dataset. However, a lot of observations lack either one or multiple variable values. So if I want ...
- 11
1
vote
1
answer
35
views
How to compare row entries in a sparse table with lots of missing values?
I have a dataset with ~1000 laptops and performance results across ~100 different benchmarks. Using the benchmark results, I want to give each laptop a single composite performance score, and rank the ...
- 113
4
votes
1
answer
788
views
Autoencoder doesn't learn 'sparse' input images
I am trying to train an autoencoder with PyTorch on 2D images containing 2D Gaussian densities such as this:
The images are of size 100x100 (I feed them into the autoencoder as 1x10000 tensors). The ...
- 41
2
votes
1
answer
723
views
What is the difference between network sparsification and model pruning
What is the difference between network sparsification and model pruning? I watched USENIX ATC '21 - Octo: INT8 Training with Loss-aware Compensation and Backward Quantization for Tiny (at 01:29sec) ...
- 243
2
votes
0
answers
102
views
What are the references for different active set selection methods for sparse Gaussian processes?
I am comparing the different sparse Gaussian process approaches within the fitrgp function in Matlab, but I am struggling to find references for the different choices within the function for the ...
- 21
0
votes
0
answers
160
views
Oracle inequality for LASSO
In chapter 6 of Statistics for High Dimensional Data (Peter BühlmannSara van de Geer), they focus on the normal linear regression model with fixed regressors and begin by stating OLS achieves a risk
$\...
- 543
2
votes
1
answer
45
views
Bayes prior in MAP estimation corresponding to $\ell^0$ penalization
I gather that in the context of penalized least squares, we can interpret a penalty term as corresponding to a prior $\pi(\beta)\propto \exp\{-\text{pen}\}.$
Is this also true for $\ell^0$ ...
- 543