The data I'm analyzing has been collected and measured from about 2'000 animals.

For every animals, I have a number of discrete values, like species, gender, colour etc., as well as some continuous values, like age, size, weight, and some medicinal values like the level of some specific chemicals in the blood.

I want to build a statistical model which can predict some missing values. For example if the data of an additional animal is missing the weight value, I want the model to predict it based on the other existing data.

Coming from a software development background, I assume that Neural Networks would be one possible solution for the problem. But there must be others.

What's the name of this kind of problem? What statistical methods are used for it?


There is a plethora of methods out there for addressing this question, but a very common method is data imputation. Here is a good Wikipedia source to get you started: http://en.wikipedia.org/wiki/Imputation_%28statistics%29 The basic premise of imputations is to construct a distribution for the missing data conditional on the actual data that you do have.

Similarly, you could try to construct the full data likelihood that incorporates both the observed and missing data, i.e.,

$$f(y|\theta)=f(y_{\text{obs}},y_{\text{mis}}|\theta)$$ where the full data $y$ is equal to the observed data $y_{\text{obs}}$ as well as the missing data $y_{\text{mis}}$. Now if you treat this problem from a Bayesian perspective then we can think of $y_{\text{obs}}$ as being a random variable and thus we can integrate out the missing data to obtain the observed data likelihood

$$f(y_{\text{obs}}|\theta)=\int_{\Theta}f(y_{\text{obs}},y_{\text{mis}}|\theta)d\theta$$ and then proceed with statistical inference using the observed data likelihood. However, the integral may be computationally expensive to deal with.

Alternatively, for conditionally independent observations the full data likelihood is $$f(y|\theta) = \prod_{i=1}^n f(y_{\text{obs},i}|\theta)\prod_{i=n+1}^{n+k}f(y_{\text{mis},i}|\theta)$$ If we once again take a Bayesian perspective and place a prior on $\theta$ then we can conduct statistical inference for $\theta$ by sampling from the posterior distribution of $\theta|y$ in a MCMC fashion. For example, one possibility would be to use a Gibbs sampler where we can sample first from $$p(\theta|y_{\text{obs}},y_{\text{mis}})$$ and then sample from $$p(y_{\text{mis}}|\theta,y_{\text{obs}})$$ repeatedly until we have a sufficient number of samples of $\theta$ to conduct inference for $\theta$.

  • $\begingroup$ Great answer. A good introduction to multiple imputation is Don Rubin (2009): Multiple Imputation for Nonresponse in Surveys. $\endgroup$ – abaumann Sep 15 '13 at 9:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.