1
$\begingroup$

How do I make partial plot (or partial dependence plot) for Gaussian Process classification? I mean plot which would show a dependence of a response probability to a single covariate variable?

Partial plots for non-linear models can be very tricky as I found. The SW package I am using only provides plots which fix the other covariates on a single fixed value (e.g., an average value of the covariates, or centroid of the "niche", peak of the response, etc.) . But I found that it depends a lot on where you fix those values. For illustration, here is the example from my GP when I fix the other covariates on (a) mean, (b) median niche (i.e. where y = 1), (c) peak (i.e. max E(y)):

enter image description here

As you can see, these are almost 3 different plots; thanks to the non-linear multi-dimensional nature of the model. It depends a lot on where you fix those values, and not a single one is guaranteed to give you a "general idea" of how the covariate works.

Thus, I think it only makes sense to average (integrate) the predictions over the distribution of the values of all the other covariates, given the particular value of the covariate of interest. In a similar manner how Gelman & Pardoe 2007 calculate APC:

$$\mbox{partialPlot}(u_1) = \int dv \int d\theta ~ \mbox{E}(y|u_1,v,\theta) \cdot p(v|u_1) \cdot p(\theta)$$

where $u$ is the covariate for which we plot the partial dependence plot, for a single value of $u_1$; $v$ are all the other covariates, $\theta$ are hyper-parameters. $\mbox{E}(y|u_1,v,\theta)$ is a single prediction.

But not only this, I would also like to know the variance of that prediction (confidence interval).

Questions:

  1. How are partial plots usually done in Gaussian Processes? Have you come across a better way than fixing the other covariates on a single value?
  2. Is my definition reasonable for partial plots in GP? Or, does it exist and already have a name?
  3. Is there any very simple R package, or a simple algorithm description, to implement a reasonable partial plot for Gaussian process? Gelman & Pardoe describe some algorithm using Mahalanobis distances but to implement this using the paper seems too complicated for me now. I guess someone had to do it already!


PS: please try to resist to write too long and too super-scientific answer; I will probably never have time (and ability) to read and understand. I got many such answers here and it's a pitty to see so much effort going another direction. Please as simple as possible, my knowledge and time for this is limited. Thanks! :-)

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.