8
$\begingroup$

As part of reproducing a model I described partially in this question on Stack Overflow, I want to obtain a plot of a posterior distribution. The (spatial) model describes the selling price of some properties as a Bernoulli distribution depending on whether the property is expensive (1) or cheap (0). In equations:

$$y_{i} \sim \text{Bernoulli}(p_{i})$$ $$p_{i} \sim \text{logit}^{-1}(b_{0} + b_{1}\text{LivingArea}/1000 + b_{2}\text{Age} + w({\bf{s}}))$$ $$w({\bf{s}}) \sim \text{MVN}({\bf{0}}, {\bf{\Sigma}}) $$

where $y_{i}$ is the binary result 1 or 0, $p_{i}$ is the probability of being cheap or expensive, $w({\bf{s}})$ is a spatial random variable where $\bf{s}$ represents its position. All of this for each $i = \{1, ..., 70\}$ because there are 70 properties in the dataset. $\bf{\Sigma}$ is a covariance matrix based on the geographical position of the data points. If you're curious about this model, the dataset can be found here.

The plot I want to obtain is the following contour plot:

enter image description here

The figure is described as "Image plot of the posterior median surface of the latent process $w({\bf{s}})$, binary spatial model". The book also says this:

Figure 5.8 shows the image plot with overlaid contour lines for the posterior mean surface of the latent $w({\bf{s}})$ process.

However, there are only 70 pairs of points in the dataset. I suppose that, in order to produce a contour plot, I need to estimate $w({\bf{s}})$ in 70*70 points. So, my question is: How do I produce this posterior median surface? So far I have samples of posterior distributions for all the parameters involved (using PyMC) and I know that I can predict $y^*$ at a new point using the posterior predictive distribution. However, I don't know how to predict values $w({\bf{s}})$ at a new point $s^*$. Maybe I'm wrong and the plot wasn't constructed by prediction but by interpolation.

UPDATE:

First, this is the median of the posterior distribution of $w({\bf{s}})$ at each location where there is a property. This is based on the MCMC trace for $w$.

enter image description here

And this is the interpolation (with a contour plot) using a radial basis function:

enter image description here

(If you're interested in the code, let me know)

As you can see, there are significant differences in the plots. A couple of questions:

  1. How can I know if these differences are explained by the interpolation procedure?

  2. Maybe, there are important variations in the posterior distribution of $w({\bf{s}})$ that I calculated and the one showed in the book. How much variation is acceptable between MCMC simulations? Even my own parameters change a bit depending on the sampling I use (Metropolis, Metropolis Adaptive.)

  3. Is there some Bayesian procedure to predict points $w(s)$ in order to generate a contour plot as I did using radial basis function?

$\endgroup$
  • 1
    $\begingroup$ Interpolation is prediction! (Because $w$ is a process, coming up with a value of $w$ at any unobserved location amounts to guessing the value of a random variable. Prediction, by definition, is guessing the value of a random variable.) $\endgroup$ – whuber Sep 5 '14 at 17:08
  • $\begingroup$ Sure. I meant to say interpolation as opposed to Bayesian prediction. By the way, I tried to use interpolation with nearest neighbor and I got awful results. $\endgroup$ – Robert Smith Sep 5 '14 at 17:21
  • $\begingroup$ You probably should get terrible results if you use the raw data in an interpolation program, because it was solving a different problem. You want a contour plot only of the $w$ term but the program was (I presume) using the $y_i$. $\endgroup$ – whuber Sep 5 '14 at 17:25
  • $\begingroup$ No, I was using the median of the posterior distribution for each $w(s)$. $\endgroup$ – Robert Smith Sep 5 '14 at 17:31
  • $\begingroup$ At which spatial locations $s$ did you compute the median of the posterior distribution? I believe the principal motivation behind running a model of this sort is to track the distribution of $w(s)$ at all points where you are interested in predicting its values, which--in the case of this contour map--would be all 4900 grid nodes. $\endgroup$ – whuber Sep 5 '14 at 17:38
1
$\begingroup$

It is very likely that the author used a Gaussian process to produce the interpolation. I think that is true because an exercise in the book describes a very similar problem to this one and requires a plot based on a Gaussian process.

I tried it and I think the resulting plot shares features with the posterior median surface of the original question. This is the median of the posterior distribution of $w(s)$ as above (it is slightly different because I ran another MCMC simulation):

enter image description here

And this is the interpolation based on a Gaussian process:

enter image description here

As you can see, the method of interpolation makes a huge difference.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.