4
$\begingroup$

I've been reading a lot recently about the concept of joint regularization in computer vision. Joint regularization builds on the observation that when learning multiple related concepts, for example "cat" and "dog" the most of the useful features to classify something as "cat" should be useful to classify something as "dog".

So a problem specific regularization term is designed. In the case I previously explained, the regularization term encourages "sharing" of useful useful features by mixing $L_1$ and $L_inf$ regularization terms. This is called joint regularization.

So, my questions are:

  1. Are there other types of problems (possibly outside of computer vision) where some type of "problem specific" regularization is used and is successful?
  2. Are there other mixes of regularization terms that have been successfully applied?
$\endgroup$
4
$\begingroup$

We did a reading group on the paper Total Variation Regularization for fMRI-Based Prediction of Behavior. The Total Variation (TV) norm is just the $L_1$ norm of the gradient, so it looks for sparse subsets (in this case brain regions) of features. Looking at it deeper, the actual implementation required a complicated form of the regulariser, as it required that the edges of the brain were respected etc.

More generally I think this is a) very common and b) the right way to approach problems, assuming that you are using optimisation methods. When you're designing a regulariser, you're effectively defining the equivalent of a Bayesian Prior. Hence you are putting domain knowledge into the problem, which allows you to infer more from your data.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.