Questions tagged [sparse]

A sparse matrix is a matrix where many of the elements are zeros. The tag can also be used for sparsity in other contexts, such as regression models with sparsity, or the "bet on sparsity"-principle.

Filter by
Sorted by
Tagged with
0
votes
0answers
7 views

Distance Metric for Sparse Data

I'm aware of similar questions like Distance metric for categorical and numerical data. I have a very sparse matrix and I want to a distance metric to find the most different items. Would performing ...
0
votes
0answers
15 views

geometric interpretation of SAFE rule for LASSO

El Ghaoui et al. showed that in the solution for the LASSO optimization problem $$ \hat{\beta} = \underset{\beta \in \mathbb{R}^p}{\operatorname{argmin}} \frac{1}{2} ||\boldsymbol{y} - \boldsymbol{X}\...
1
vote
0answers
16 views

Is K-medoids / partitioning around medoids (PAM) appropriate for clustering data with many zero values?

I need to cluster a matrix which contains zero values. I am clustering three separate sets of 24 values. The first two are non-zero and represent hourly ambient temperature (in K) and electrical ...
1
vote
1answer
26 views

How to train a Neural Network on sparse data?

I am trying to train a sequence model to extract specific substrings. I am working on extremely sparse text data (Sparsity ~ 0.03%, <1000 examples). After training for 500 epochs, the performance ...
0
votes
0answers
18 views

Calculating similarities between two populations using embeddings

I would like to find items from population B that are most similar to an item from population A. I have the following set up: Two sparse datasets where each row is an item (treat row index as item ID)...
2
votes
1answer
32 views

Power simulation on glmer.nb gave strange results

I would like to ask for solution or advice on strange result that glmer.nb from lme4 generated when simulating using simR package. I’m working on longitudinal gut microbiome abundance data (23 ...
0
votes
0answers
17 views

Bootstrapping in logistic regression with sparse binary variable

I would like to estimate the probability of a relation between two entities. The data set includes information of many relations between many entities, and information on covariates for each entity. ...
0
votes
1answer
17 views

Using prior knowledge about correlated variable in ridge regression

I am wondering what methods are available for incorporating prior knowledge of some variable that is correlated with the unknown regression coefficients in a ridge regression. I have a sparse matrix ...
0
votes
0answers
10 views

How to control the level of sparsity on sgPLS and spls packages?

I am trying to use a (group) sparse PLS algorithm on a regression problem with an univariate response variable $y$, and I found the packages sgPLS and ...
0
votes
0answers
22 views

How to force auto encoder learn sparse data?

My data is sparse consisting of only zeros and a few ones. Since majority of values are zero, an auto encoder leans toward outputting just zeros. How can I force the network to learn the cells ...
2
votes
1answer
33 views

Why is computation of scores in Sparse PCA different from T=XP?

I am recently learning Sparse PCA. From a lately published paper All sparse PCA models are wrong, but some are useful. Part I: Computation of scores, residuals and explained variance I learned that ...
1
vote
0answers
17 views

For ridge regression, show if $K$ columns of $X$ are identical then we must have same corresponding parameters

Show if $K$ columns of $X$, $({X_{j1}, X_{j2}...X_{jk}}) $are identical then we must have $\hat\beta_{j1},\hat\beta_{j2},...\hat\beta_{jk} $ are same in the ridge regression: $$\hat\beta = \underset{...
0
votes
1answer
32 views

Compact encoding (vectorization) of unbounded sets

Question I have a set of sets. Each set is unbounded. I would like to find a methodology to encode (vectorize) each subset. I am more specifically interested in memory efficient solutions. ...
1
vote
1answer
21 views

Why atoms in the dictionary of Dictionary Learning method are not required to be orthogonal?

According to Sparse Dictionary Learning (wiki), Sparse dictionary learning is a representation learning method which aims at finding a sparse representation of the input data (also known as sparse ...
1
vote
0answers
48 views

Sparse PCA for $p >> n$ solution with Elastic Net

I was reading about the sparse principal component approach by Zou, Hastie and Tibshirani but I do not quite understand how they handle the $p \gg n$ case in their paper. To derive the sparse axis, ...
2
votes
1answer
29 views

How to conduct k-fold cross-validation with spare outcomes?

In many applications of machine learning, the outcome vector is sparse - e.g. containing millions of 0s and a handful of 1s. When the outcome vector is sparse, many of the training sets in k-fold ...
0
votes
0answers
25 views

Correaltion between sparse variables

I have an Events x People matrix M, where a cell (e,p) gives the score of person j at event e. Let E be the total number of events. Each person has attended a lot of events, say 0.3*E at an average. ...
2
votes
0answers
119 views

How is explained variance in sparse PCA calculated?

Sparse PCA is a technique proposed by Zou et all in this paper. In usual PCA the obtained loadings are orthonormal, and the resulting scores are uncorrelated. However, in sparse PCA you give up these ...
0
votes
0answers
59 views

Using Pearson correlation coefficient in sparse data

I have been using the cor function in R to compare correlation between my variables. The data did pretty poorly with 2/3 of them having a correlation close to 1 with some other variable. However the ...
7
votes
1answer
574 views

What does Sparse PCA implementation in Python do?

I am interested on using sparse PCA in python and I found the sklearn implementation. However, I think this python implementation solves a different problem than the original sparse pca algorithm ...
0
votes
0answers
23 views

How does Data Augmentation work for supervised learning models?

I've ran into a few Kaggle competitions where the winning solution used data augmentation, and a new ML platform, which claimed to help with Data Augmentation. Use cases were imbalanced classes and ...
1
vote
1answer
33 views

Computation of generalized least squares solutions of large sparse systems

Suppose $X$ and $\Omega$ are large sparse matrices, with $\Omega$ symmetric positive definite (but not diagonal), and $y$ is a vector. I want to find the generalized least squares solution: $$\hat\...
0
votes
0answers
8 views

method for determing the important factor for high dimension categorical data

I have around 1000 people with total 400 categorical features, but each one will only have subset of those 400 features(ranging from 3-60 for this population), thus the dataset is fairly sparse. Now I ...
3
votes
2answers
715 views

Shrinkage priors

I am building a Bayesian model where I to put shrinkage priors such as spike and slab, horseshoe prior, etc on some parameters for feature selection, but I am not able to decide which one is the best. ...
1
vote
2answers
187 views

What are the benefits of sparse representations and sparse parameters?

What are their benefits? I know sparse parameters are a different story than sparse representations, but I want to know how each of these can benefit us and which one is more important than the other ...
0
votes
0answers
34 views

How to get adjusted explained variance of PC using Sparse PCA in R

I am using "elasticnet" package to do the sparse PCA. However I could only get the percent of explained variance of each PC. I am wondering can I get the adjusted explained variance of each PC using "...
0
votes
0answers
54 views

Extracting the right summary statistics from zero-inflated data sets (i.e. a sparse matrix where everything non-zero is a statistical outlier)

I'm a consumer tech startup founder with rudimentary background in statistics. I need help in processing a large, sparse matrix. I'm logging all actions users are undertaking in my app. I then ...
0
votes
0answers
40 views

Comparing clusterings of different spaces

I have developed a graph embedding method that learns feature representations for each node in the graph based on the graph topology. I want to demonstrate that the learned feature vectors yield ...
2
votes
0answers
27 views

How to interpret the learning curve of a LinearRegression for sparse data?

I have a dataset of shape ca.(4800, 350). Both the dataset X as well as the response y is very sparse (ca. 3500 samples with y=0). I wanted to take a look at the learning curve to estimate the bias-...
3
votes
1answer
100 views

Sparse linear poorly constrained least-squares problem

I have a somewhat simple linear problem. I have data $D$ (a vector with a few million elements), the parameter vector $X$ (a couple of thousands elements) and the design matrix $A$ which is extremely ...
3
votes
1answer
88 views

Combining many sparse binary variables

Based on kjetil b halvorsen suggestion, I rephrased my problem: My problem is analogous to the following: i am supposed to predict if a high school student will go to university (Yes/No). I have ...
0
votes
1answer
35 views

Nonlinear approximation of a sparse linear transformation

Suppose that we have $y = Ax$ where $x$ is a vector of size $m \times 1$, $A$ is a sparse matrix with size $n \times m$. Suppose that $n \times m$ is very large and we wish to find a non-linear ...
0
votes
0answers
54 views

How to build a machine learning model from sparse features with class imbalance?

I have around 10 numerical features and 1 class/target (e.g., visitors count of a website). All of them are sparse. Sparsity is around %70-80. The median of the class/target is zero. Is there a good ...
0
votes
0answers
8 views

Appropriate machine learning technique for spectral data and low-frequency feedback

I have a performance measure and a data source that basically supplies a complex and varying waveform. I would like to apply some unstructured learning technique to try and find a pattern in the ...
0
votes
1answer
232 views

Clustering Binary and Continuous Features

If you need to cluster a dataset with the following characteristics: It has a mix of binary and continuous features. It is very sparse. For most features, you only have values for 15% of the ...
0
votes
0answers
27 views

Why neural network does not learn sparse relationship [duplicate]

The universal approximation theorem says that a neural network can be used to approximate any continuous function under some regular conditions with arbitrarily small approximation error, provided we ...
2
votes
0answers
24 views

Probability of sparse spectrum

Consider a vector $v$ such that $v \sim \mathrm{Unif}(\mathbb{S}^{d-1})$, the uniform distribution on the unit sphere in $d$ dimensions. Question: is there an upper bound on the probability that $v$ ...
1
vote
0answers
36 views

Is it useful to use sparse regression (e.g. Lasso) when the number of observations is significantly larger than the number of covariates?

I'm learning about penalized/sparse regression and I noticed that the examples used for penalized/sparse regression, e.g. Lasso, are usually cases where the number of observations is significantly ...
2
votes
0answers
104 views

How to conduct a principal component analysis on data set with large number of zeros

I have data for percentage cover of plant species in 500 sites. There are columns for 30 different species in the data set and I would like to drastically reduce this down to a manageable number of ...
2
votes
0answers
135 views

Does anyone know the rank of the Netflix Prize dataset?

I'm looking into the Netflix Prize at the moment. We model the dataset as an $n \times m$ matrix, where $n$ is the number of users and $m$ is the number of movies. Does anyone know the rank of the ...
0
votes
2answers
673 views

Autoencoder for sparse data

Suppose I have a big (1,000x20,000) sparse (95% of elements are zeros) matrix with counts. I want to use autoencoder to encode-decode this matrix. How should I do it? Are there any tricks or ...
3
votes
1answer
171 views

estimate precision matrix with given spatial sparsity pattern

I have a set of $n$ measurements of $p$ variables $\xi_i$. I am interested in the inverse covariance or precision matrix $P$ of the variables, but because $p \gg n$ and because of limited storage ($p$ ...
1
vote
1answer
315 views

LSTM time series forecasting on sparse dataset

I am working on the LSTM time series forecasting of solar energy production. The available data is one year on a half hourly basis. More than 60% of the data values are zero as the PV stations cannot ...
0
votes
1answer
50 views

Longitudinal study - generalised linear mixed model - dealing with very wide confidence intervals due to sparsity in the outcome

I am conducting a treatment evaluation using administrative data. It is a population-based study of all people diagnosed with a specific disorder in two calendar years (N = 2300). I have run a GLMM ...
6
votes
1answer
127 views

Literature on $\ell_q$ LASSO, $q < 1$

I am not sure how is $\ell_q$-LASSO called, but here I am talking about LASSO regression, with $\| \beta \|_{\ell_q}$ regularization, $q< 1$. In popular literature, such as Elements of Statistical ...
0
votes
0answers
16 views

How to develop features for deep learning from cart items data?

I wonder how to approach building set of features to feed deep learning model (eg convnet) from cart items data: 5pcs of product1 1pcs of product5 2pcs of product8 Assuming 30-50 products per ...
2
votes
0answers
34 views

Is low rank finite-iteration manifold identification possible?

In sparse optimization, I am trying to solve the problem $$ \min_{x\in \mathbb R^{n}} \quad f(x) + \|x\|_1 $$ and at optimality, $x^*$ may be sparse. If I define the sparse manifold as $\mathcal M = ...
1
vote
1answer
58 views

finding sparse regions in time series data

I have several hundred years of church baptisms that will be searched by people wanting to find the baptisms of their ancestors. I want to call attention to periods in the records in which the number ...
2
votes
1answer
3k views

Do you standardize the data before PCA whitening?

I have a data set ranged in different scales as well as some variables are sparse, for example, ...
1
vote
0answers
128 views

Normalizing sparse matrix by mean, should the mean be calculated excluding zero?

I have very sparse matrix (70% sparsity) which I want to normalize by mean. I tried using mean both include and exclude zero. The histogram between count (y-axis) and value (x-axis) shows The value ...

1
2 3 4 5