Statistical Analysis on Sparse data? I had few data analysis and modelling roles and was always "unfortunate" to work with, let's call it bad data. Think of default probabilities 1 of 10 000, or sales time series that has 3 sold products per day which occasionally jumps to 10 or drops to 0 with certain seasonal component of course.
What do you think, can one even to proper analysis on these kinds of sparse data? If yes, what techniques would you recommend?
 A: Pretty sketchy question but answerable nonetheless. The answer is "yes," there is lots one can do with sparse data. This response is far from "complete" but will review a few options in a kind of "DIY" shotgun listing. In other words, it is up to the analyst to decide which option may be appropriate to pursue.
The first consideration is to identify where the sparsity is occurring, e.g., is it in the features in terms of a large, complex, combinatorial set of possibilities or is it wrt the target or dependent variable which may have few observed responses, or both?
Wrt sparse or rare events in the target variable, e.g., when response to a stimulus is recorded as 0,1 or "yes/no," and the response rate is very small, one common error is to model this using logistic regression. The mistake is this: it is well known that the logistic curve does not provide a good fit to the tails of its distribution. This means that with sparse or rare event data, logistic regression will produce biased results. Commonly recommended "solutions" for this problem are to go out and get a larger sample of data or, alternatively, to specifically subsample those segments that are both important to the analysis and sparsely populated. This is a bad idea for at least two reasons: one, it's not always possible to simply "get more data" and two, even when possible, it can be prohibitively costly in terms of time and money. Better solutions are possible:


*

*Gary King, Harvard political scientist and statistical methodologist, discusses rare event analysis... http://gking.harvard.edu/category/research-interests/methods/rare-events. 

*Quantification and prediction of extreme events in a one-dimensional
nonlinear dispersive wave model ... http://sandlab.mit.edu/Papers/14_PhysicaD.pdf

*https://www.analyticsvidhya.com/blog/2014/01/logistic-regression-rare-event/
Concerning features, one needs to distinguish between structural zeros which occur for logically impossible combinations and sparsity where a combination of features is possible, there just isn't enough information to populate that particular cell in the table. Consider healthcare or hospital data where a combination such as male patients given a diagnostic code of "pregnancy" is possible from a purely computational point of view in terms of cross-classifying a set of features, but males actually giving birth is impossible, i.e., it is to be considered a structural zero. But sex and gender are different constructs. So, until transgender patients (e.g., female to male gender) have children, this will remain a structural zero. 
As noted, sparsely populated features are different, requiring special tools to facilitate analysis from those employed for target variables. The following is a "laundry list" or shotgun set of options for dealing with sparse features. Much of it was gathered by simply browsing for the keywords "inference from sparse data." Choose carefully from the list:


*

*Adopt a Bayesian modeling framework. For instance, Gelman and Hill state in chapter 13 of their book Data Analysis Using Regression and Multilevel/Hierarchical Models that it is possible to analyse features with a sample size of 1. Frequentists might object to this claim. MCMC sampling provides a workaround for sparsely populated categorical features in that pooling data across the sampling iterations builds a distribution about a feature in the posterior, even in cases where there is a sample size of 1. 

*Gelman, in his blog, also discusses sparsity here ... http://andrewgelman.com/2013/12/16/whither-the-bet-on-sparsity-principle-in-a-nonsparse-world/.

*Alan Agresti, Approximate Is Better than "Exact" for Interval Estimation of Binomial Proportions ...http://www.stat.ufl.edu/~aa/articles/agresti_coull_1998.pdf

*Data mining for rare events ... http://www-users.cs.umn.edu/~aleks/pakdd04_tutorial.pdf

*comparison of different methods for modelling rare events data ... http://lib.ugent.be/fulltxt/RUG01/002/163/708/RUG01-002163708_2014_0001_AC.pdf

*Statistical Inference Methods for Sparse Biological Time Series Data https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3114728/

*Bayesian non-parametric models and inference for sparse and hierarchical latent structure ... http://cs.stanford.edu/people/davidknowles/daknowles_thesis.pdf

*Statistical Inference: n-gram Models over Sparse Data .. http://www.sims.berkeley.edu/~jhenke/Tdm/TDM-Ch6.ppt

*Lost in a random forest: Using Big Data to study rare events ... http://journals.sagepub.com/doi/pdf/10.1177/2053951715604333
And so on.
Good luck.
