Questions tagged [sparse]
A sparse matrix is a matrix where many of the elements are zeros. The tag can also be used for sparsity in other contexts, such as regression models with sparsity, or the "bet on sparsity"-principle.
259
questions
1
vote
1answer
906 views
How to find lasso beta estimates
I'm following this paper https://arxiv.org/pdf/1304.4773.pdf
And for the moment I'm just trying to go through the steps for equation $(1.2)$
$$ \hat \beta_{lasso} = argmin_{\beta} \sum_{i}^{n} (y_{i}...
1
vote
0answers
200 views
How to impute a highly sparse binary matrix?
Let's say you begin with a binary matrix (only 0 and 1) and you must impute 60% missing values.
How would you do this? It is possible to use nearest neighbors?
4
votes
1answer
696 views
Sparse PCA using elasticnet package in R - How to know how many number of nonzero values in one PC?
Can someone help me on Sparse PCA? I am using the "elasticnet" package to perform sparse PCA. I am having a hard time in figuring out how many nonzero values should a component contain?
For example, ...
0
votes
2answers
2k views
How to implement density-based clustering?
I’m looking to implement density-based clustering with R or Mathematica on a giant file (600,000 points on a 3 billion x 3 billion plane).
Is DBSCAN the right method for data that is this sparse? I am ...
6
votes
2answers
618 views
What is the advantage of sparsity?
In many problems the solution is sparse and the authors of methods usually present it as an advantage. How could one leverage the sparsity in general? Why is an advantage?
6
votes
2answers
1k views
Random forest and LASSO regression both give different variable importances
I have a dataset with 163 observations (all countries in the world with population > 1000000) and 290 variables related to their disease burden and performance. Because there are more variables than ...
1
vote
1answer
223 views
Plausbility of parameter estimates in a 2PLM IRT model: A case of inherently sparse data
This question relates to my previous query regarding differences in IRT parameter estimates (2PLM model) between the ltm package in R and the Mplus latent variable program which is linked here: 2PLM ...
3
votes
1answer
174 views
Extension to SAFE screening rule for Lasso
In El Ghaoui et al. (2010), "Safe feature elimination in sparse learning" and following works, screening rules are derived for Lasso (as well as other L1-penalized problems):
$ \min_w \|y-X w\|^2 + \...
4
votes
1answer
2k views
How To Deal With Large Numbers Of Categorical Predictors
I have three data sets that, when joined, have O(320) independent variables for a classification problem.
Principal component analysis (PCA) seems out of the question because the data is mostly ...
1
vote
2answers
6k views
Linear regression on large sparse feature set
I have a sparse feature matrix with 50K observations and 150K features. All features are binary. On this I have to run a linear regression. I want just a decent fit.
Data:
Let us consider training ...
1
vote
0answers
220 views
Regularized cox survival model with time varying covariates and sparce matrix in R
I was wondering if there is a survival framework in R (or any other language for that matter) for doing the following:
Fitting an extended (i.e., time-varying covariates) cox survival model
...
3
votes
1answer
7k views
what is sparse solution?
This graph is about panelty least square method, and it is said that once a parameter hits zero, it remains zero for larger garmma, and this results in a sparse solution. I am wondering what it means ...
2
votes
0answers
80 views
Are there kernel-based one-class sparse kernel-based outlier detection methods, e.g. one-class Relevance Vector Machine?
I have a commercial outlier detection problem in moderate dimension (8-25).
We have a limited number of true positive tags and can roughly evaluate performance of various methods. So far, the 1-...
7
votes
3answers
534 views
Goodness of fit test on sparse contigency tables with high dimensionality
I have a vector of size 1x3500, which can be viewed as the 'known distribution'. It is simply a table of counts across 3500 groups (i.e. a contingency table). I also have $N$ other vectors of the same ...
3
votes
1answer
242 views
Compressed Sensing: Missing Fourier Coefficients?
This question is regarding the problem of reconstructing a signal given only a subset of the Fourier coefficients are observed:
$$\min_x \|x\|_1 \text{ subject to } y = Ax$$
where $x = (x_1,x_2,\...
1
vote
0answers
29 views
Showing a bound on the $L_2$ error in the N-sparse approximation of a vector
This is a supposedly 'trivial bound' from Donoho's Compressed Sensing paper - trying to figure out where it comes from.
Assume that $\theta$ is a vector that obeys the following constraint for some $...
7
votes
3answers
939 views
Invert a sparse covariance matrix
I have a postive definite symmetric covariance matrix which looks like this:
Note that all A,B,C,D,E,F,G are also poitive definite symmetric covariance matrices I want to find an easy way were I can ...
6
votes
1answer
10k views
User segmentation by clustering with sparse data
Imagine that I have 100k users and 1k categories. For each user, up to 5 categories, I know how much money they have spent. Obviously my data is very sparse.
Now I want to group users by the money ...
1
vote
1answer
1k views
Dimensionality reduction for high dimensional sparse data before clustering or spherical k-means?
I am trying to build my first recommender system where i create a user feature space and then cluster them into different groups. Then for the recommendation to work for a particular user , first i ...
4
votes
2answers
184 views
Theoretical justification for bag of words
Bag of words and visual bag of words are successful machine learning approaches. Does anyone know of a theoretical justification for why / when they work?
What I am trying to explain is the good ...
1
vote
1answer
182 views
what is the method in dictionary learning which does not have a overcomplete dictionary?
what is the method in dictionary learning which does not have a overcomplete dictionary? and what is the difference in minimization between these two methods (one using overcompelte dictionary and ...
1
vote
1answer
1k views
Best way to handle sparse + non-sparse data to create a model
I'm wondering what is the best way to handle sparse+non-sparse data in e.g. a Ridge regression using scikit learn.
Ridge can handle both sparse and nonsparse data.
Imagine something simple as a ...
1
vote
1answer
886 views
efficient “dot” product of two sparse vectors with uncertainty
I'm interested in extensions or analogs of the vector dot product that apply to sparse vectors in the case of uncertainty in the abscissa.
The vectors I deal with are often of large (100,000 or 1,...
5
votes
2answers
7k views
Gradient boosting decision tree implementation
I am willing to implement my own GBM. I have been looking - unsuccessfully - for a clear article describing the implementation of gradient boosting machine for decision trees. Sources like this are ...
2
votes
0answers
181 views
Clustering algorithm advice for extracting key features in sparse data
I have the following dataset: consider a dataset $X$ of $1400 \times 600$. The rows represent households at time $1 \leq t \leq 14$. So I have $100$ households. The columns represent the programs that ...
2
votes
0answers
585 views
Miss Forest & Iterative PCA : How to handle very sparse matrix imputation?
I am currently benchmarking matrix completion methods (k-NN, RandomForest and iterative PCA) on multivariate normal data in which I introduce a certain proportion of NA (5 to 95%). My performance ...
3
votes
0answers
3k views
Typical range of values for TFIDF
I am working on a text corpus. Each line contains between 10 and 50 words. There are around 25 000 words in the whole text and 1 000 000 lines. I turned this corpus into its tf-idf representation.
I ...
11
votes
1answer
4k views
Is large scale PCA even possible?
Principal component analysis' (PCA) classical way is to do it on an input data matrix which columns have zero mean (then PCA can "maximize variance"). This can be achieved easily by centering the ...
2
votes
2answers
811 views
Finding pattern from sparse matrix
I have a large sparse matrix which represents whether the action is happened or not. Each columns represent each action. Each row represents time. The data is collected for every 15min. Zero in the ...
2
votes
0answers
589 views
How to go from sparse matrix to linear regression model (using SVD)?
I am trying to replicate the Kosinski, Stillwell, & Graepel (2013) study about predicting private traits and attributes from Facebook like data for study purposes. First I have admit, however, ...
7
votes
2answers
188 views
How to prove the properties of penalized likelihood estimator in Fan and Li (2001) paper
I'm reading through Fan and Li (2001) Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties. On p. 1349 (near the bottom-right corner) they proposed three properties that a ...
1
vote
0answers
272 views
Kernel K nearest neighbours with sparse data
I have a big sparse matrix (around 5 million of lines, 20 000 predictors), and I would like to run a kernelized k-NN on it. However, I don't know how to scale the data properly.
So far, I have scaled ...
7
votes
0answers
823 views
Feature scaling for non-negative sparse data
Imagine you have many observations on which you want to run a classification algorithm. Each observation is characterized by a matrix of non-negative values. For all observations 90-98% of the values ...
0
votes
1answer
1k views
What's the best (Google chart) visualisation for displaying sparse timeline data across thousands of “columns”
I am trying to visualise a sparse dataset but am finding it hard to fit it into the standard categories of charts. I'm a developer building with Google Charts and I really want to stick with that ...
5
votes
1answer
165 views
Does the opposite of nested cross-validation make sense?
I'm asking the question from a machine learning point of view. I have a dataset with relatively high sparsity, so if I use nested cross-validation for my feature tuning and evaluation, that is tune ...
2
votes
0answers
107 views
Instances of sparse covariance matrices
I am trying to find large datasets with inherently sparse covariance matrices, to be tested with our algorithm. Basically, we will take the sample covariance matrix and enforce some structured ...
3
votes
0answers
410 views
sparse covariance/correlation thresholding
In our project, we would like to do some optimization on sparse matrices. The idea is to scrape massive amounts of data, form a covariance/correlation matrix, and form a sparsity pattern basically by ...
0
votes
1answer
183 views
What are the techniques to deal with classifying sparse categorical features?
Suppose I have a group of features each one is sparse with a few number of values (1-10) what are the required preprocessing steps required to avoid degradation of the performance of the classifier ...
0
votes
0answers
475 views
Problems using pcr (from pls library) in R with large number of qualitative variables
I'm trying to classify a variable into either 0 or 1, using 50 factors, with a sample size of 2000. 25% of the dependent variables are 0 and the rest are 1.
Of these factors, 30 are categorical. I've ...
0
votes
0answers
312 views
Handling sparse document term matrix
I am working with a corpus of several thousand documents (41,732) however the documents tend to be short (the median number of terms per document is 3) resulting in a sparse document term matrix.
...
2
votes
2answers
188 views
Spectral norm of a sparse Gaussian matrix
Suppose $G$ is an $m \times n$ matrix such that each entry of $G$ is a standard normal variable. We know that the spectral norm of $G$ scales as $\sqrt m + \sqrt n$. Now, given a set of indices $S$ ...
1
vote
0answers
100 views
Individual factor significance in multilevel sPLS-DA
I recently was asked by reviewers to "include p-values" with my multilevel sparse partial least squares analysis. In brief, I have a nested design with two factors, say treatment and sampling region. ...
2
votes
2answers
140 views
How to convert the objective function to canonic form of sparse coding?
As we know the conventional sparse coding problem (LASSO) is:
$\min_{\alpha} \| X-D\alpha\|_F^2 + \lambda \|\alpha\|_{1} \tag{1}$
where $X$ , $D$, and $\alpha$ are data, dictionary and coefficients ...
9
votes
2answers
1k views
Generate symmetric positive definite matrix with a pre-specified sparsity pattern
I am trying to generate a correlation matrix $p\times p$ (symmetric p.s.d) with a pre-specified sparsity structure (specified by a graph on $p$ nodes). The nodes that are connected in the graph have ...
1
vote
1answer
288 views
Calculating means and confidence intervals when most data points are 0
I am looking at data set that has four groups. In each group, the data is mostly, 99+% of time, composed of zeros, but, when it is not zero it can be any float number (e.g., 0.01 to 921.2, with most ...
3
votes
2answers
5k views
How to build a predictive model with a billion of sparse features?
I am making a model to learn a dataset which has a big feature number and sparse samples (I am planning to use logistic regression). The feature number can be as big as 1,000,000,000. It is sparse ...
1
vote
2answers
3k views
Kernel SVM on sparse data
I have a sparse dataset where a lot of the columns (features) contain mostly zero values. Class labels are multiple discrete categories (10 classes to be precise). I'm wondering if this should trouble ...
8
votes
2answers
3k views
Logistic regression with sparse predictor variables
I am currently modeling some data using a binary logistic regression. The dependent variable has a good number of positive cases and negative cases - it is not sparse. I also have a large training set ...
9
votes
1answer
750 views
What guidelines should be followed for using Neural Networks with sparse inputs
I have extremely sparse inputs, e.g. locations of certain features in an input image. Further each feature can have multiple detections (not sure if this will have a bearing on the design of the ...
7
votes
1answer
4k views
Cross correlation for very sparse binary data
I have a very large (5271159x60) sparse (~2.5%) binary matrix, and I'd like to calculate the cross correlation between each of the columns (sensors) for a series of lags from -10:10, which would give ...
