Questions tagged [sparse]

A sparse matrix is a matrix where many of the elements are zeros. The tag can also be used for sparsity in other contexts, such as regression models with sparsity, or the "bet on sparsity"-principle.

Filter by
Sorted by
Tagged with
2 votes
1 answer
20 views

How to use sparse PCA loadings in a regression?

I'm using Dr Frank Harrell's code in RMS 2nd edition. He goes into sparse PCA. Does anyone know how to code a regression model after getting the sparse component grid? ...
user avatar
  • 21
2 votes
0 answers
25 views

Fitting Sparsed Constrained regression with non-negative coefficients adding to 1

I see a similar problem in How do I fit a constrained regression in R so that coefficients total = 1? Specifically, my model is $Y_i= \pi_1 X_1+\pi_2 X_2 +...+ \pi_K X_K +\epsilon_i$ with $\pi_k \ge 0$...
user avatar
0 votes
0 answers
19 views

T-testing on sparse data

Problem: I have two stochastic processes, $S_1$ and $S_2$, that frequently are zero, but occasionally have positive values with unknown probabilities $q_1$ and $q_2$. e.g. $$ S_1 = \{0,0,0,0,0,21,0,0,...
user avatar
  • 1
1 vote
1 answer
57 views

Why lasso cannot be arbitrarily applied?

Consider any log likelihood function $f(\theta|x)$ where $x$ is data. I can consider $f(\theta|x)+\lambda||\theta||_1$ where $||\theta||_1$ is the standard $L_1$ norm. It seems that I cannot apply ...
user avatar
  • 839
1 vote
2 answers
113 views

How to solve the problem of having sparse data that would become too small when aggregated?

I have a dataset that provides the count of cyber incidents since 2011 for different countries and different attack types, and I want to use this data in a machine learning model to predict future ...
user avatar
1 vote
1 answer
67 views

Is it wrong to run a Random Forest on high-dimensional, sparse, and unbalanced data?

I am learning about random forests, and I have been testing using R. I have doubts about whether I am doing something wrong given that my data are: sparse, high-dimensional, and unbalanced. Trying to ...
user avatar
1 vote
0 answers
18 views

Working with sparse data in numpy and sklearn [closed]

I have a time series dataset geenrated from some electrophysiological data. I have a frequncy dataset and the matrix is quite sparse but huge, like it contains 0.005 s time bins for 2000+ neurons ...
user avatar
0 votes
0 answers
47 views

bootstrapped l2,1 least square does not produce sparse solution

I am trying to model an autoregressive model with $\ell$-2,1 regularization, where $\|X\|_{2,1}=\sum_i|\sum_jX_{ij}|$: $$y_{t+1} = wx_{t} + \lambda_1\|w\|_{2,1}, y\in \mathbb{R}^{n_1}, x\in \mathbb{R}^...
user avatar
  • 231
0 votes
1 answer
38 views

Variable selection with sparse data

I have a dataset with 141 observations and 8 corresponding variables and I mean to apply a GLM to this dataset. However, a lot of observations lack either one or multiple variable values. So if I want ...
user avatar
  • 11
0 votes
0 answers
10 views

What is the degree of cell sparsity that Poisson model can tolerate?

For models with no interaction terms, my understanding is that all the marginal cells need to have non-zero counts, in order to have finite estimates. If this minimum requirement is satisfied, will a ...
user avatar
  • 475
1 vote
1 answer
22 views

How to compare row entries in a sparse table with lots of missing values?

I have a dataset with ~1000 laptops and performance results across ~100 different benchmarks. Using the benchmark results, I want to give each laptop a single composite performance score, and rank the ...
user avatar
4 votes
1 answer
80 views

Autoencoder doesn't learn 'sparse' input images

I am trying to train an autoencoder with PyTorch on 2D images containing 2D Gaussian densities such as this: The images are of size 100x100 (I feed them into the autoencoder as 1x10000 tensors). The ...
user avatar
1 vote
1 answer
31 views

What is the difference between network sparsification and model pruning

What is the difference between network sparsification and model pruning? I watched USENIX ATC '21 - Octo: INT8 Training with Loss-aware Compensation and Backward Quantization for Tiny (at 01:29sec) ...
user avatar
  • 189
2 votes
0 answers
38 views

What are the references for different active set selection methods for sparse Gaussian processes?

I am comparing the different sparse Gaussian process approaches within the fitrgp function in Matlab, but I am struggling to find references for the different choices within the function for the ...
user avatar
  • 21
0 votes
0 answers
24 views

Fit a histogram of sparse data with a monotonically decreasing function

I have a business problem where essentially we have counts across a certain metric which at times suffer from low observation counts! So essentially when you look across the bins you'll have adequate ...
user avatar
  • 21
0 votes
0 answers
43 views

Compatibility condition in LASSO

I am reading Statistics for High-Dimensional Data (Bühlmann and van de Geer). Chapter 6 discusses obtaining the oracle inequality in LASSO under the compatibility condition, a technical assumption ...
user avatar
0 votes
0 answers
39 views

Oracle inequality for LASSO

In chapter 6 of Statistics for High Dimensional Data (Peter BühlmannSara van de Geer), they focus on the normal linear regression model with fixed regressors and begin by stating OLS achieves a risk $\...
user avatar
2 votes
1 answer
30 views

Bayes prior in MAP estimation corresponding to $\ell^0$ penalization

I gather that in the context of penalized least squares, we can interpret a penalty term as corresponding to a prior $\pi(\beta)\propto \exp\{-\text{pen}\}.$ Is this also true for $\ell^0$ ...
user avatar
3 votes
1 answer
90 views

Bayesian priors associated with regularization penalties

I gather that adding a penalty term to (linear) least squares minimization typically corresponds with choosing some prior for Bayes estimation in the normal linear regression model. A couple questions ...
user avatar
0 votes
0 answers
27 views

LASSO with $L_p$ norms for $1 < p < 2$?

For the sparse linear regression problem, minimizing the LASSO objective $\| X \beta - Y \|_2^2 + \lambda \| \beta \|_1$ is known to recover the optimal data generating parameter $\beta^*$ with the ...
user avatar
0 votes
0 answers
81 views

regarding the loss function for the sparse autoencoder

I am implementing a sparse autoencoder using dictionary learning framework, i.e. instead of using a neural network in Keras, Pytorch or Tensorflow, I am using only matrices to represent the different ...
user avatar
  • 1,586
0 votes
0 answers
20 views

Doubts about generating a synthetic dataset according to a paper

I'm trying to replicate the experiment reported in section 3.3 of this paper (https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2930825/) but I'm struggling to understand how the synthetic dataset is ...
user avatar
0 votes
0 answers
7 views

Do we need to shrink bias term in a sparse linear model?

In a sparse linear model, we put shrinking constraint on parameters associated with predictors only as the following optimization problem: $minimize_{\omega_0, \omega} (1/2n (\|y-\omega_0 \textbf{1} - ...
user avatar
0 votes
0 answers
64 views

binary one dimensional k-svd

I have a problem that is somehow between k-means and k-svd: I would like to find a vector $D=[d_1, d_2, d_3..., d_k]$, so that the elements of another vector $Y=[y_1,y_2,y_3..., y_n]$ with $k \ll n$ ...
user avatar
  • 1
2 votes
1 answer
65 views

If there is a sparse solution then is the smallest l1 norm solution at least as sparse?

Consider the linear equation $Ax=b$ where $A$ is a matrix and $b$ and $x$ are vectors. Suppose there exists a vector $x_S$ that solves this equation ($Ax_S=b$) and $x_S$ has $k$ entries of value $0$ (...
user avatar
0 votes
0 answers
39 views

Multiple regression with sparse X

I would like to run the multiple regression: $$y = X \beta + \epsilon \,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,(1)$$ with: $n \sim 2 \times 10^6$ $p \sim 10^4$ $X$ sparse 70% of rows have 1 non-...
user avatar
2 votes
1 answer
49 views

In a sparse reward problem, is it possible to remove reward shaping once the RL agent trains long enough to consistently reach the final reward?

I'm new to machine learning, and primarily looking at it from the perspective of it's applications to control theory. In the application found in this paper, a RL agent attempts to land a spacecraft ...
user avatar
0 votes
0 answers
20 views

Issues in having high-dimensional and sparse data

I was wondering about the issues one would encounter in a Machine Learning algorithm having data represented by high-dimensional vectors that are also sparse. In particular, I know that having many ...
user avatar
0 votes
0 answers
19 views

Recommender System without Ratings but Duration instead

I'm currently working on a recommender system without ratings variable. I only have the watch duration for streamers and I should be able to recommend a list of streamers with importance on its order. ...
user avatar
0 votes
0 answers
74 views

LARS package in R with specific lambda value

I am trying to use the LARS package in R to obtain a Lasso estimate of the sparse coefficient vector, say $\hat{\beta}_{\text{sparse}}$, as opposed to a coefficient path. In other words, I am not ...
user avatar
  • 1,030
1 vote
0 answers
58 views

outlier detection for sparse data in categorical variable

I have a big dataset with a column "clientid" and a categorical column "choice". I want to find out what are the clients that have strange combinations of choices (less frequent ...
user avatar
0 votes
0 answers
39 views

Calculate Data Sparsity - There are no 'zeros'

I am being asked to provide the percentage of data sparsity in my dataset. The challenge I am running into is that there isn't traditional sparsity in my data, no 'zeros', 'NAs', '-' etc.. I am ...
user avatar
3 votes
0 answers
177 views

Clustering text embeddings: TF-IDF + BERT Sentence Embeddings

I am trying to cluster a few thousand forum posts that are similar in content to Stackoverfow. So far, I have tried two main approaches to represent the posts: TF-IDF Sentence embedding based on BERT....
user avatar
  • 41
2 votes
1 answer
278 views

Scaling a sparse matrix

I want to apply sparse PCA to a sparse matrix. I was wondering if scaling to mean 0 and unit variance would be appropriate given that my input is sparse?
user avatar
  • 21
7 votes
1 answer
199 views

Probability distribution associated with nuclear norm?

The $\ell_1$ norm of model parameters is often added to loss functions because it induces sparsity in the solution of the overall cost function: $$ c(\theta) = -\log L(x|\theta) + \lambda ||\theta||_1$...
user avatar
  • 1,832
2 votes
0 answers
73 views

Are Spiking Neural Networks The Next Big Thing? [closed]

Intel recently announced their Loihi chip as part of their "Neuromorphic Computing" research, which is optimized for spiking neural networks (SNNs). I found an example of a problem in which ...
user avatar
0 votes
0 answers
65 views

Reconstruction metric robust to scaling, sparsity, and outliers?

I seek a reconstruction error metric with following properties: Robustness to sparsity: error decreases in presence of many zeros or small values (if predicted correctly) Scale invariance: error ...
user avatar
1 vote
1 answer
209 views

unsupervised anomaly detection on sparse data

Given that I have a very sparse data matrix with continuous features, like this dataframe for example ...
user avatar
4 votes
0 answers
191 views

What is the appropriate metric for determining distance / dissimilarity of sparse, high dimensional data in PCA space?

I'm working with scRNA-seq data (~96% sparse, high dimensional), and am trying to determine distances between the cells in PCA space - NOT for the specific purpose of clustering. The principal ...
user avatar
2 votes
2 answers
52 views

L1 feature selection followed by exhaustive search

I'm working with a group on an ML project and one of the team members has proposed using L1 to reduce the feature space and then apply an exhaustive search with the reduced feature set. In each step, ...
user avatar
0 votes
0 answers
91 views

NLP multiclass classification with many sparse classes

I am attempting to use natural language processing to geocode "addresses". The address is the result of a write-in of a survey where the respondent is instructed to give their city, state, ...
user avatar
  • 1,408
1 vote
1 answer
158 views

Estimation of Sparse Panel Data

There are 1000 students and 100 teachers. Each teacher is given the answer scripts of randomly selected 100 students. So in total 10,000 answer scripts are judged. Now this is sort of panel data, but ...
user avatar
  • 63
1 vote
1 answer
151 views

What is a Dirac distribution on a hyperplane?

I'm trying to understand message passing for compressed sensing. I came acrross this distribution: As the title suggests, how does this distribution look like? I know the first products term in the ...
user avatar
  • 97
1 vote
0 answers
59 views

Is K-medoids / partitioning around medoids (PAM) appropriate for clustering data with many zero values?

I need to cluster a matrix which contains zero values. I am clustering three separate sets of 24 values. The first two are non-zero and represent hourly ambient temperature (in K) and electrical ...
user avatar
  • 11
1 vote
1 answer
344 views

How to train a Neural Network on sparse data?

I am trying to train a sequence model to extract specific substrings. I am working on extremely sparse text data (Sparsity ~ 0.03%, <1000 examples). After training for 500 epochs, the performance ...
user avatar
  • 159
2 votes
1 answer
225 views

Power simulation on glmer.nb gave strange results

I would like to ask for solution or advice on strange result that glmer.nb from lme4 generated when simulating using simR ...
user avatar
0 votes
1 answer
116 views

Using prior knowledge about correlated variable in ridge regression

I am wondering what methods are available for incorporating prior knowledge of some variable that is correlated with the unknown regression coefficients in a ridge regression. I have a sparse matrix ...
user avatar
  • 3
2 votes
1 answer
105 views

Why is computation of scores in Sparse PCA different from T=XP?

I am recently learning Sparse PCA. From a lately published paper All sparse PCA models are wrong, but some are useful. Part I: Computation of scores, residuals and explained variance I learned that ...
user avatar
  • 68
1 vote
0 answers
98 views

For ridge regression, show if $K$ columns of $X$ are identical then we must have same corresponding parameters

Show if $K$ columns of $X$, $({X_{j1}, X_{j2}...X_{jk}}) $are identical then we must have $\hat\beta_{j1},\hat\beta_{j2},...\hat\beta_{jk} $ are same in the ridge regression: $$\hat\beta = \underset{...
user avatar
0 votes
1 answer
41 views

Compact encoding (vectorization) of unbounded sets

Question I have a set of sets. Each set is unbounded. I would like to find a methodology to encode (vectorize) each subset. I am more specifically interested in memory efficient solutions. ...
user avatar

1
2 3 4 5 6