0
$\begingroup$

Much like this post: Quadratic Programming and Lasso, I'm trying to integrate RIDGE Penalty in a dedicated quadratic solver. In my case, I am working with quadprog from MATLAB. Unlike LASSO where you can eliminate the absolute value in the constrained form and rewrite them in linear form (effectively keeping a quadratic problem), you can't with RIDGE. This means that in order to have a quadratic problem, I have to work with the penalty form:

$$ RIDGE: \sum_{i=1}^{N} (y - x'\beta)^2 + \lambda \sum \beta_{i}^{2}$$

My explicit problem is to minimize the variance with added RIDGE Penalty.

$${\underset{w}{\arg\min}} \frac{1}{2} w' \Sigma w \ + \lambda \sum w_i^{2}$$ $$s.t. \ \sum_{i=1}^{N} w_i = 1$$

Basically, I want to minimize the variance while summing the weights to 1. A pretty standard problem in finance. My question is: How to adapt the objective function so that it includes the penalty? When working with a dedicated solver like quadprog, you can only specify the positive definite squared matrix and the vector for the unsquared terms. With the formulation below, you then specify $H$ and $f$. Link: https://www.mathworks.com/help/optim/ug/quadprog.html

$${\underset{x}{\arg\min}} \frac{1}{2} x' H x \ + f'x$$

I can either modify H (which is my covariance matrix), but this would change the number of values in my $w$ vector, or I could work with $f'$, but this is for unsquared term. I need to implement $\lambda x'x$ in my objective function, which is equal to $\lambda \sum x_i^{2}$.

EDIT: I forgot to mention, but $\lambda$ is arbitrarily given. I will optimize for multiple values. Cross-validation has been considered, but it's complicated for time-series and I might just choose multiple values myself.

Improve this question
$\endgroup$
7
  • $\begingroup$ What are the relationships between $\omega, \beta$, and $B$? As written, the penalty term doesn't affect the optimal value of $\omega$ at all... $\endgroup$
    – jbowman
    Oct 7, 2020 at 16:32
  • $\begingroup$ @jbowman Thanks for noticing. It was my mistake. It might be confusing, but the first is purely a definition and is in regression form. It usually is given with $\beta$. As for my own definition, the $w$ is the weight vector. I changed it in the definition. The third definition uses $x$ because I wanted to keep the same variables used on the quadprog page. In my case, the $x$ vector is my $w$ vector. $\endgroup$
    – Samuel Normandeau
    Oct 7, 2020 at 16:37
  • $\begingroup$ I don't follow exactly what you're doing but it's best not to force the problem to fit into a quadratic optimization solver. Rather than doing this, you can use the more general approach of writing the objective as a function and then using some non-linear solver such as BFGS or Rvmin. See what's available by doing ?optim if you use R. $\endgroup$
    – mlofton
    Oct 7, 2020 at 17:11
  • $\begingroup$ @mlofton The alternative (Lagrangian form) would be to remove $\lambda \sum w_i^{2}$ and put a constraint of the form $\sum w_i^{2} \leq s$, however this becomes a non-linear problem. My professor wants me to work with the penalty formulation to use a quadratic solver. It's not really forcing since there is an equivalence between the two forms. It's just two different approaches. Basically, I'm trying to minimize the variance of my portfolio with the added penalty term in the objective function. The concept is simple, but I can't adapt my function to the quadprog solver. $\endgroup$
    – Samuel Normandeau
    Oct 7, 2020 at 17:17
  • 1
    $\begingroup$ ... as, given $\lambda$, you are minimizing $(1/2) \omega'(\Sigma + 2\lambda I)\omega$. $\endgroup$
    – jbowman
    Oct 7, 2020 at 18:23

1 Answer 1

Reset to default
0
$\begingroup$

Who is your professor? The model he has assigned comes from the following paper:

  • de Miguel et al (2009) A Generalized Approach to Portfolio Optimization: Improving Performance by Constraining Portfolio Norms

Instead of using an additive penalty term, the ridge shrinkage of the portfolio weight vector should, or works best, as a separate constraint:

$${\underset{w}{\arg\min}} \frac{1}{2} w' \Sigma w \ $$

\begin{aligned} s.t. & \sum w_i^{2} < \delta \\ & \ \sum_{i=1}^{N} w_i = 1 \end{aligned}

where $\delta$ has a one-to-one inverse correspondence to $\lambda$. In other words, instead of increasing $\lambda$ to make the portfolio weights smaller, you decrease $\delta$ to achieve the same regularization effect.

This is what your professor means by adapting the objective function for the penalty. The linear regression formula first shown is better suited to the Lagrangean approach to regularization, whereas the optimization (second) formula you showed is better suited to the constrained optimization approach of regularization, and also deflects concerns of non-linear optimization since the main objective function (portfolio variance) I wrote is quadratic as is, while the two constraints are linear. Both approaches are equivalent due to the one-to-one correspondence between $\lambda$ and $\delta$.

If you insist on using the additive $\lambda$ penalty term, then the objective would reduce to the well-known closed-form analytical solution for the ridge covariance matrix where $I$ is an identity matrix the size of $\Sigma$.

$${\underset{w}{\arg\min}} \frac{1}{2} w'(\Sigma + 2\lambda I)w$$ $$s.t. \ \sum_{i=1}^{N} w_i = 1$$

Improve this answer
$\endgroup$
8
  • $\begingroup$ I might misunderstand, but are you saying $\sum w_i^2 \leq \delta$ is a linear constraint that can be written in the form $Ax \leq b$ effectively keeping the program quadratic? And my problem with the ridge covariance matrix modification is: say I start looping over some increasing target return to get the efficient frontier, there won't be any explicit constraints to prevent me from getting large allocations. Therefore, instead of converging towards 0 (like Ridge should do as $s$ goes down or $\lambda$ goes up, numbers could explode as nothing seems to be constraining them. $\endgroup$
    – Samuel Normandeau
    Oct 7, 2020 at 19:30
  • $\begingroup$ the concern of ridge exploding has no substance or relevance to a possible target portfolio return constraint because the ridge covariance matrix is known to be more stable and well-conditioned than the sample $\Sigma$. And being overly compliant with the $Ax=b$ rule is interfering with the practicality of the above well-known model. If matlab, or your limited understanding of that rule with respect to constraints, is holding you back, then consider moving to python's scipy.optimize.minimize. The entire portfolio norm exercise tends to be straightforward and unremarkable to be honest $\endgroup$
    – develarist
    Oct 7, 2020 at 19:37
  • $\begingroup$ I understand what you are saying. While using fmincon, I can easily adapt your model (which I have done already), my problem only comes when trying to use quadprog which only allows constraints of the type $Ax \leq b$ and $Ax = b$ (and which my professor wants me to use for some reason -- "use a dedicated quadratic solver"). Thanks for your help! $\endgroup$
    – Samuel Normandeau
    Oct 7, 2020 at 19:42
  • $\begingroup$ could you rephrase the constraint types by replacing the $A$, $x$ and $b$s with the actual variables shown in the answer instead? if so, they are linear as already said $\endgroup$
    – develarist
    Oct 7, 2020 at 19:52
  • $\begingroup$ Well, in my case $A = 1_N$, $b = s$ and $x = w$, but since $w$ is squared, clearly it isn't linear to me and I can't express it in matrices of the form $Aw \leq s$. $\endgroup$
    – Samuel Normandeau
    Oct 7, 2020 at 20:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.