I would like to use multiple class logistic regression to learn the decision boundaries separating the different classes (denoted by color) in the image below. Kernel logistic regression with a RBF kernel seems like a good choice, but I would like the decision boundary, when projected back to the 2-d space, to fall along the white grid lines. One way of proceeding would be to introduce a penalty term in the objective function, but I'm not sure how to proceed. Does anyone have any references for using a grid as a contraint in this kind of problem?

A perhaps different way of asking this is how can I combine, on the one hand, a prior belief of the form $\Pr(\int^\mathbf{x}\pi=.5 \mid \mathbf{x})$ about the distribution over $\mathbf{x}$ of when the cumulative probability distribution of $\pi$ takes on a particular value with, on the other hand, a likelihood $\Pr(\mathbf{x} \mid \pi)$? I

Different classes of points on a grid

  • 1
    $\begingroup$ Stupid question -- can't you just partition the space into regions cut by those lines and count dots inside each? $\endgroup$
    – user88
    Commented Nov 6, 2011 at 10:07
  • $\begingroup$ Great question. However, the gradient at the decision boundary is a parameter of interest, and I would like to also be able to have a continuous measure of how likely a given location is to have a certain label. $\endgroup$
    – fgregg
    Commented Nov 6, 2011 at 15:10

1 Answer 1


Just to rephrase this: this appears to merely involve fitting GLMs over local partitions, with partitioning granularity essentially already defined. There is a maximum granularity (i.e. the white boxes) and the path from the minimum granularity of partitioning to the maximum is defined by splits that produce the boxes (i.e. somewhat less granular than every box would be a combination of local boxes). Is that correct?

If this is correct, then you can define a decision tree with logistic regression performed on leaf nodes. Partitioning splits involve membership in a box or an aggregate of boxes. If the number of boxes is small, you can decide whether to do bottom-up aggregation or top-down splitting. Within each region, just do logistic regression.

FWIW, this is an interesting problem, with the possibility of extensions to spatial modeling.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.