We changed our privacy policy. Read more.

Questions tagged [sparse]

A sparse matrix is a matrix where many of the elements are zeros. The tag can also be used for sparsity in other contexts, such as regression models with sparsity, or the "bet on sparsity"-principle.

Filter by
Sorted by
Tagged with
0
votes
0answers
21 views

Is there a method for summarising a dataset in terms of "sparsity" and "uniformity"?

My apologies if I'm using the wrong terms here or the question has been asked before but I couldn't find an answer to my questions. I have a dataset with 7 continuous variables (6 features and a ...
0
votes
0answers
37 views

Multiple regression with sparse X

I would like to run the multiple regression: $$y = X \beta + \epsilon \,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,(1)$$ with: $n \sim 2 \times 10^6$ $p \sim 10^4$ $X$ sparse 70% of rows have 1 non-...
2
votes
1answer
18 views

In a sparse reward problem, is it possible to remove reward shaping once the RL agent trains long enough to consistently reach the final reward?

I'm new to machine learning, and primarily looking at it from the perspective of it's applications to control theory. In the application found in this paper, a RL agent attempts to land a spacecraft ...
0
votes
0answers
12 views

Issues in having high-dimensional and sparse data

I was wondering about the issues one would encounter in a Machine Learning algorithm having data represented by high-dimensional vectors that are also sparse. In particular, I know that having many ...
0
votes
0answers
15 views

Recommender System without Ratings but Duration instead

I'm currently working on a recommender system without ratings variable. I only have the watch duration for streamers and I should be able to recommend a list of streamers with importance on its order. ...
0
votes
0answers
42 views

LARS package in R with specific lambda value

I am trying to use the LARS package in R to obtain a Lasso estimate of the sparse coefficient vector, say $\hat{\beta}_{\text{sparse}}$, as opposed to a coefficient path. In other words, I am not ...
0
votes
0answers
36 views

Score Function for Sparse Mean Gaussian?

The Sparse Mean Gaussian model can be described by $ X \sim \mathcal{N}(\theta, \sigma^2 I_d)$ where $ \vert\vert\theta\vert\vert_{0} = s$ where $d$ is the dimension of the random variable, and $s$ ...
0
votes
0answers
5 views

Is spare encoding a special case of spare autoencoding with ignoring non linear activation

we know that Sparse encoding is to minimize the objective function: $$\sum\limits_{n=1}^N\Big(||x_n - Az_n||^2 + \lambda\rho(z_n)\Big).$$ Here $A = [a_1,\cdots,a_M]...
0
votes
0answers
64 views

Spatial modelling of sparse count data

I am trying to model claim counts over for a given region. The data is very sparse. I am using the BESAG model from R INLA package. I am having a tough time to model the data. I am able to reproduce ...
1
vote
0answers
35 views

outlier detection for sparse data in categorical variable

I have a big dataset with a column "clientid" and a categorical column "choice". I want to find out what are the clients that have strange combinations of choices (less frequent ...
0
votes
0answers
29 views

Calculate Data Sparsity - There are no 'zeros'

I am being asked to provide the percentage of data sparsity in my dataset. The challenge I am running into is that there isn't traditional sparsity in my data, no 'zeros', 'NAs', '-' etc.. I am ...
2
votes
0answers
53 views

Clustering text embeddings: TF-IDF + BERT Sentence Embeddings

I am trying to cluster a few thousand forum posts that are similar in content to Stackoverfow. So far, I have tried two main approaches to represent the posts: TF-IDF Sentence embedding based on BERT....
0
votes
0answers
17 views

Difference between Structural Topic Modeling(STM) and SAGE (Sparse Additive Generative Model)?

I have read that STM combines 3 models of: (1) correlated topic model (CTM) (2) Dirichlet-Multinomial Regression (DMR) topic model (3) Sparse Additive Generative Model (SAGE) Is it correct to just ...
0
votes
0answers
13 views

Machine Learning algorithms for sparse database

I have a very sparse database with more than 200000 rows (instances) and 500 columns that lead to almost 100 million entries. However, only 205000 of the data are non zero, that is almost 0.2% of the ...
0
votes
0answers
14 views

Comparing sparse vectors

I am looking for a metric for comparing gene count tables. These are long columns of data (a few millions genes by a few dozen samples), with all non-negative entries, about 90% of which are zeros. ...
0
votes
0answers
25 views

Why can the L1 Norm be expressed as a constrain?

I am learning why the L1 regulariser (Lasso) is used to encourage sparsity in ML models. When describing the proof, I am seeing that the regularised minimisation cost function; $$ min(RSS(w) + \lambda*...
2
votes
1answer
104 views

Scaling a sparse matrix

I want to apply sparse PCA to a sparse matrix. I was wondering if scaling to mean 0 and unit variance would be appropriate given that my input is sparse?
4
votes
0answers
51 views

Probability distribution associated with nuclear norm?

The $\ell_1$ norm of model parameters is often added to loss functions because it induces sparsity in the solution of the overall cost function: $$ c(\theta) = \log L(x|\theta) + \lambda ||\theta||_1$$...
2
votes
0answers
55 views

Are Spiking Neural Networks The Next Big Thing? [closed]

Intel recently announced their Loihi chip as part of their "Neuromorphic Computing" research, which is optimized for spiking neural networks (SNNs). I found an example of a problem in which ...
0
votes
0answers
44 views

Reconstruction metric robust to scaling, sparsity, and outliers?

I seek a reconstruction error metric with following properties: Robustness to sparsity: error decreases in presence of many zeros or small values (if predicted correctly) Scale invariance: error ...
1
vote
1answer
160 views

unsupervised anomaly detection on sparse data

Given that I have a very sparse data matrix with continuous features, like this dataframe for example ...
4
votes
0answers
111 views

What is the appropriate metric for determining distance / dissimilarity of sparse, high dimensional data in PCA space?

I'm working with scRNA-seq data (~96% sparse, high dimensional), and am trying to determine distances between the cells in PCA space - NOT for the specific purpose of clustering. The principal ...
2
votes
2answers
38 views

L1 feature selection followed by exhaustive search

I'm working with a group on an ML project and one of the team members has proposed using L1 to reduce the feature space and then apply an exhaustive search with the reduced feature set. In each step, ...
0
votes
0answers
68 views

NLP multiclass classification with many sparse classes

I am attempting to use natural language processing to geocode "addresses". The address is the result of a write-in of a survey where the respondent is instructed to give their city, state, ...
1
vote
1answer
125 views

Estimation of Sparse Panel Data

There are 1000 students and 100 teachers. Each teacher is given the answer scripts of randomly selected 100 students. So in total 10,000 answer scripts are judged. Now this is sort of panel data, but ...
1
vote
1answer
135 views

What is a Dirac distribution on a hyperplane?

I'm trying to understand message passing for compressed sensing. I came acrross this distribution: As the title suggests, how does this distribution look like? I know the first products term in the ...
1
vote
0answers
43 views

Is K-medoids / partitioning around medoids (PAM) appropriate for clustering data with many zero values?

I need to cluster a matrix which contains zero values. I am clustering three separate sets of 24 values. The first two are non-zero and represent hourly ambient temperature (in K) and electrical ...
1
vote
1answer
168 views

How to train a Neural Network on sparse data?

I am trying to train a sequence model to extract specific substrings. I am working on extremely sparse text data (Sparsity ~ 0.03%, <1000 examples). After training for 500 epochs, the performance ...
2
votes
1answer
154 views

Power simulation on glmer.nb gave strange results

I would like to ask for solution or advice on strange result that glmer.nb from lme4 generated when simulating using simR ...
0
votes
1answer
75 views

Using prior knowledge about correlated variable in ridge regression

I am wondering what methods are available for incorporating prior knowledge of some variable that is correlated with the unknown regression coefficients in a ridge regression. I have a sparse matrix ...
2
votes
1answer
79 views

Why is computation of scores in Sparse PCA different from T=XP?

I am recently learning Sparse PCA. From a lately published paper All sparse PCA models are wrong, but some are useful. Part I: Computation of scores, residuals and explained variance I learned that ...
1
vote
0answers
55 views

For ridge regression, show if $K$ columns of $X$ are identical then we must have same corresponding parameters

Show if $K$ columns of $X$, $({X_{j1}, X_{j2}...X_{jk}}) $are identical then we must have $\hat\beta_{j1},\hat\beta_{j2},...\hat\beta_{jk} $ are same in the ridge regression: $$\hat\beta = \underset{...
0
votes
1answer
38 views

Compact encoding (vectorization) of unbounded sets

Question I have a set of sets. Each set is unbounded. I would like to find a methodology to encode (vectorize) each subset. I am more specifically interested in memory efficient solutions. ...
1
vote
1answer
67 views

Why atoms in the dictionary of Dictionary Learning method are not required to be orthogonal?

According to Sparse Dictionary Learning (wiki), Sparse dictionary learning is a representation learning method which aims at finding a sparse representation of the input data (also known as sparse ...
2
votes
0answers
85 views

Sparse PCA for $p >> n$ solution with Elastic Net

I was reading about the sparse principal component approach by Zou, Hastie and Tibshirani but I do not quite understand how they handle the $p \gg n$ case in their paper. To derive the sparse axis, ...
2
votes
1answer
77 views

How to conduct k-fold cross-validation with spare outcomes?

In many applications of machine learning, the outcome vector is sparse - e.g. containing millions of 0s and a handful of 1s. When the outcome vector is sparse, many of the training sets in k-fold ...
0
votes
0answers
53 views

Correaltion between sparse variables

I have an Events x People matrix M, where a cell (e,p) gives the score of person j at event e. Let E be the total number of events. Each person has attended a lot of events, say 0.3*E at an average. ...
4
votes
0answers
506 views

How is explained variance in sparse PCA calculated?

Sparse PCA is a technique proposed by Zou et all in this paper. In usual PCA the obtained loadings are orthonormal, and the resulting scores are uncorrelated. However, in sparse PCA you give up these ...
7
votes
1answer
2k views

What does Sparse PCA implementation in Python do?

I am interested on using sparse PCA in python and I found the sklearn implementation. However, I think this python implementation solves a different problem than the original sparse pca algorithm ...
2
votes
1answer
42 views

Computation of generalized least squares solutions of large sparse systems

Suppose $X$ and $\Omega$ are large sparse matrices, with $\Omega$ symmetric positive definite (but not diagonal), and $y$ is a vector. I want to find the generalized least squares solution: $$\hat\...
4
votes
2answers
1k views

Shrinkage priors

I am building a Bayesian model where I to put shrinkage priors such as spike and slab, horseshoe prior, etc on some parameters for feature selection, but I am not able to decide which one is the best. ...
1
vote
2answers
657 views

What are the benefits of sparse representations and sparse parameters?

What are their benefits? I know sparse parameters are a different story than sparse representations, but I want to know how each of these can benefit us and which one is more important than the other ...
0
votes
0answers
99 views

Extracting the right summary statistics from zero-inflated data sets (i.e. a sparse matrix where everything non-zero is a statistical outlier)

I'm a consumer tech startup founder with rudimentary background in statistics. I need help in processing a large, sparse matrix. I'm logging all actions users are undertaking in my app. I then ...
2
votes
0answers
43 views

How to interpret the learning curve of a LinearRegression for sparse data?

I have a dataset of shape ca.(4800, 350). Both the dataset X as well as the response y is very sparse (ca. 3500 samples with y=0). I wanted to take a look at the learning curve to estimate the bias-...
3
votes
1answer
105 views

Sparse linear poorly constrained least-squares problem

I have a somewhat simple linear problem. I have data $D$ (a vector with a few million elements), the parameter vector $X$ (a couple of thousands elements) and the design matrix $A$ which is extremely ...
3
votes
1answer
327 views

Combining many sparse binary variables

Based on kjetil b halvorsen suggestion, I rephrased my problem: My problem is analogous to the following: i am supposed to predict if a high school student will go to university (Yes/No). I have ...
0
votes
1answer
54 views

Nonlinear approximation of a sparse linear transformation

Suppose that we have $y = Ax$ where $x$ is a vector of size $m \times 1$, $A$ is a sparse matrix with size $n \times m$. Suppose that $n \times m$ is very large and we wish to find a non-linear ...
0
votes
0answers
119 views

How to build a machine learning model from sparse features with class imbalance?

I have around 10 numerical features and 1 class/target (e.g., visitors count of a website). All of them are sparse. Sparsity is around %70-80. The median of the class/target is zero. Is there a good ...
0
votes
1answer
566 views

Clustering Binary and Continuous Features

If you need to cluster a dataset with the following characteristics: It has a mix of binary and continuous features. It is very sparse. For most features, you only have values for 15% of the ...
0
votes
0answers
34 views

Why neural network does not learn sparse relationship [duplicate]

The universal approximation theorem says that a neural network can be used to approximate any continuous function under some regular conditions with arbitrarily small approximation error, provided we ...

1
2 3 4 5 6