Using R, I can only find tools for performing L1 and/or L2 regularized linear regression (lars, glmnet) and tools for constrained linear regression (quadprog , or lsei {limSolve} , where the inequality and equality constraints can be only given in the form Ax = b , Gx <= h).

It seems inutitive for me that the possibility of combining both should be required very oft when solving specific regression problems, but so far I havent been able to do it.

Instead of providing information on my specific set of constraints and algebraic system, I´d be interested to know if this is a problem I can actually solve using the above mentioned packages? Are there any packages at all in R built for both parameter regularization and specific parameter penalties?

Update: For better understanding: I am not trying to combine different regularization methods (like in elastic net), nor trying to combine different parameter constraints. My goal is to combine regularization with specific coefficient constraints, so for example:

Find the most sparse solution (penalizing absolute values through LASSO) of a linear regression y = bx which satisfy the coefficient constraints bA < h for some given matrix A and threshold h.

min($\parallel$ $\beta$x-y$\parallel_2^2$ + $\lambda$ $\mid$ $\beta$ $\mid_1$ ), s.t $\beta$ A $\leqslant$ h

  • 3
    $\begingroup$ Stan lets you estimate any model you can dream of, at the cost of a steeper learning curve. $\endgroup$
    – Sycorax
    Apr 7 '16 at 13:44
  • $\begingroup$ Thank you for the fast answer. Yes I have heard of rstan as a very complete modeling package. I just thought my problem is so basic, it should be possible to modifiy practically any linear regression tool in order to set specific constraints and a penalized norm to the estimated set. $\endgroup$
    – hirschme
    Apr 7 '16 at 13:52
  • $\begingroup$ Actually, now that you mention it, I think that I answered a similar question like this before. There was a trick that would let you coerce the linear constraint to just be a regular linear model.... it's maybe a year old. I don't have time to look now, but perhaps you'd be able to find it. $\endgroup$
    – Sycorax
    Apr 7 '16 at 13:56
  • 2
    $\begingroup$ Presuming that you already know MATLAB, it is very simple to specify and solve problems of the type you have described using CVX cvxr.com/cvx . $\endgroup$ Apr 8 '16 at 15:28
  • 1
    $\begingroup$ @Mark Thank you for your advise, it was really easy to implement my problem using CVX in matlab. I guess for this kind of tasks I will be moving away from R more into the wolfram alpha and mathworks world.. $\endgroup$
    – hirschme
    Apr 12 '16 at 12:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.