0
$\begingroup$

Can any one recommend some reference or advice for getting a (hard) thresholding rule combining $\lambda_1$ and $\lambda_2$ for solving an optimization problem with the two penalty (regularization) terms? Maybe fused Lasso is an example.

Thanks in advance!

$\endgroup$
  • $\begingroup$ So your question is: how to determine the values of weight penalties? $\endgroup$ – Hossein Apr 20 '17 at 15:50
  • $\begingroup$ @Hossein Hmmm I should say yes $\endgroup$ – i_a_n Apr 20 '17 at 15:55
0
$\begingroup$

Just as two hints:

  • Empirically evaluate different weight penalties using cross-validation. That is, divide your data into the training and validation sets and try different weight penalties to train the model on the training set and choose those values that have the best performance on the validation set.

  • Use the method proposed by David MacKay to empirically obtain the values for weight penalties without cross-validation. Take a look at this coursera course for more detail:

    Geoffrey Hinton. "Lecture 9.6- MacKays Quick and Dirty Method of Setting Weight Costs." COURSERA: Neural networks for machine learning (2012).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.