Skip to main content
Tweeted twitter.com/StackStats/status/1430364559764860934
edited body
Source Link
Sycorax
  • 94k
  • 23
  • 236
  • 390

I couldn't find a better title, but here's the thing...

I was studying Elastic Net regularization and I found this function: $$ \text{Loss} = \sum_{i=0}^n \left(y_i - (wx_i + c)\right)^2 + \lambda_1 \sum_{i=0}^{m-1} \left|w_i\right| + \lambda_2 \sum_{i=0}^{m-1} w_i^2 \\ $$$$ \text{Loss} = \sum_{i=0}^n \left(y_i - (wx_i + c)\right)^2 + \lambda_1 \sum_{j=0}^{m-1} \left|w_j\right| + \lambda_2 \sum_{j=0}^{m-1} w_j^2 \\ $$

Sorry about the image, but it's much easier this way. So I found this as being the loss function. However, there is no lambda 1, nor lambda 2 in Scikit-Learn. Instead, we find alpha and l1_ration. After studying more, I found that Scikit-Learn alpha is actually lambda and Scikit-Learn l1_ratio is actually alpha, both from this other equation:

$$ L_\text{enet} = \frac{1}{2n}\sum_{i=1}^n (y_i - x_i^T \beta)^2 + \lambda \left( \frac{1-\alpha}{2} \sum_{j=1}^m \beta_j^2+ \alpha \sum_{j=1}^m \left|\beta_j\right| \right) $$

So I guess the biggest question is: how do I go from the first equation to the last equation? How are these two equations connected?

I couldn't find a way to go from the first equation to the second one.

Again, sorry about the image, I know this isn't the best practice, but it was so much easier to just add them in here.

NOTE: Please consider w and beta as the same thing, the coefficients from the regression.

I couldn't find a better title, but here's the thing...

I was studying Elastic Net regularization and I found this function: $$ \text{Loss} = \sum_{i=0}^n \left(y_i - (wx_i + c)\right)^2 + \lambda_1 \sum_{i=0}^{m-1} \left|w_i\right| + \lambda_2 \sum_{i=0}^{m-1} w_i^2 \\ $$

Sorry about the image, but it's much easier this way. So I found this as being the loss function. However, there is no lambda 1, nor lambda 2 in Scikit-Learn. Instead, we find alpha and l1_ration. After studying more, I found that Scikit-Learn alpha is actually lambda and Scikit-Learn l1_ratio is actually alpha, both from this other equation:

$$ L_\text{enet} = \frac{1}{2n}\sum_{i=1}^n (y_i - x_i^T \beta)^2 + \lambda \left( \frac{1-\alpha}{2} \sum_{j=1}^m \beta_j^2+ \alpha \sum_{j=1}^m \left|\beta_j\right| \right) $$

So I guess the biggest question is: how do I go from the first equation to the last equation? How are these two equations connected?

I couldn't find a way to go from the first equation to the second one.

Again, sorry about the image, I know this isn't the best practice, but it was so much easier to just add them in here.

NOTE: Please consider w and beta as the same thing, the coefficients from the regression.

I couldn't find a better title, but here's the thing...

I was studying Elastic Net regularization and I found this function: $$ \text{Loss} = \sum_{i=0}^n \left(y_i - (wx_i + c)\right)^2 + \lambda_1 \sum_{j=0}^{m-1} \left|w_j\right| + \lambda_2 \sum_{j=0}^{m-1} w_j^2 \\ $$

Sorry about the image, but it's much easier this way. So I found this as being the loss function. However, there is no lambda 1, nor lambda 2 in Scikit-Learn. Instead, we find alpha and l1_ration. After studying more, I found that Scikit-Learn alpha is actually lambda and Scikit-Learn l1_ratio is actually alpha, both from this other equation:

$$ L_\text{enet} = \frac{1}{2n}\sum_{i=1}^n (y_i - x_i^T \beta)^2 + \lambda \left( \frac{1-\alpha}{2} \sum_{j=1}^m \beta_j^2+ \alpha \sum_{j=1}^m \left|\beta_j\right| \right) $$

So I guess the biggest question is: how do I go from the first equation to the last equation? How are these two equations connected?

I couldn't find a way to go from the first equation to the second one.

Again, sorry about the image, I know this isn't the best practice, but it was so much easier to just add them in here.

NOTE: Please consider w and beta as the same thing, the coefficients from the regression.

added 248 characters in body
Source Link
Sycorax
  • 94k
  • 23
  • 236
  • 390

I couldn't find a better title, but here's the thing...

I was studying Elastic Net regularization and I found this function: Elastic net loss function$$ \text{Loss} = \sum_{i=0}^n \left(y_i - (wx_i + c)\right)^2 + \lambda_1 \sum_{i=0}^{m-1} \left|w_i\right| + \lambda_2 \sum_{i=0}^{m-1} w_i^2 \\ $$

Sorry about the image, but it's much easier this way. So I found this as being the loss function. However, there is no lambda 1, nor lambda 2 in Scikit-Learn. Instead, we find alpha and l1_ration. After studying more, I found that Scikit-Learn alpha is actually lambda and Scikit-Learn l1_ratio is actually alpha, both from this other equation:

elastic net alternative loss function$$ L_\text{enet} = \frac{1}{2n}\sum_{i=1}^n (y_i - x_i^T \beta)^2 + \lambda \left( \frac{1-\alpha}{2} \sum_{j=1}^m \beta_j^2+ \alpha \sum_{j=1}^m \left|\beta_j\right| \right) $$

So I guess the biggest question is: how do I go from the first equation to the last equation? How are these two equations connected?

I couldn't find a way to go from the first equation to the second one.

Again, sorry about the image, I know this isn't the best practice, but it was so much easier to just add them in here.

NOTE: Please consider w and beta as the same thing, the coefficients from the regression.

I couldn't find a better title, but here's the thing...

I was studying Elastic Net regularization and I found this function: Elastic net loss function

Sorry about the image, but it's much easier this way. So I found this as being the loss function. However, there is no lambda 1, nor lambda 2 in Scikit-Learn. Instead, we find alpha and l1_ration. After studying more, I found that Scikit-Learn alpha is actually lambda and Scikit-Learn l1_ratio is actually alpha, both from this other equation:

elastic net alternative loss function

So I guess the biggest question is: how do I go from the first equation to the last equation? How are these two equations connected?

I couldn't find a way to go from the first equation to the second one.

Again, sorry about the image, I know this isn't the best practice, but it was so much easier to just add them in here.

NOTE: Please consider w and beta as the same thing, the coefficients from the regression.

I couldn't find a better title, but here's the thing...

I was studying Elastic Net regularization and I found this function: $$ \text{Loss} = \sum_{i=0}^n \left(y_i - (wx_i + c)\right)^2 + \lambda_1 \sum_{i=0}^{m-1} \left|w_i\right| + \lambda_2 \sum_{i=0}^{m-1} w_i^2 \\ $$

Sorry about the image, but it's much easier this way. So I found this as being the loss function. However, there is no lambda 1, nor lambda 2 in Scikit-Learn. Instead, we find alpha and l1_ration. After studying more, I found that Scikit-Learn alpha is actually lambda and Scikit-Learn l1_ratio is actually alpha, both from this other equation:

$$ L_\text{enet} = \frac{1}{2n}\sum_{i=1}^n (y_i - x_i^T \beta)^2 + \lambda \left( \frac{1-\alpha}{2} \sum_{j=1}^m \beta_j^2+ \alpha \sum_{j=1}^m \left|\beta_j\right| \right) $$

So I guess the biggest question is: how do I go from the first equation to the last equation? How are these two equations connected?

I couldn't find a way to go from the first equation to the second one.

Again, sorry about the image, I know this isn't the best practice, but it was so much easier to just add them in here.

NOTE: Please consider w and beta as the same thing, the coefficients from the regression.

added 93 characters in body
Source Link
Yuxxxxxx
  • 327
  • 2
  • 6

I couldn't find a better title, but here's the thing...

I was studying Elastic Net regularization and I found this function: Elastic net loss function

Sorry about the image, but it's much easier this way. So I found this as being the loss function. However, there is no lambda 1, nor lambda 2 in Scikit-Learn. Instead, we find alpha and l1_ration. After studying more, I found that Scikit-Learn alpha is actually lambda and Scikit-Learn l1_ratio is actually alpha, both from this other equation:

elastic net alternative loss function

So I guess the biggest question is: how do I go from the first equation to the last equation? How are these two equations connected?

I couldn't find a way to go from the first equation to the second one.

Again, sorry about the image, I know this isn't the best practice, but it was so much easier to just add them in here.

NOTE: Please consider w and beta as the same thing, the coefficients from the regression.

I couldn't find a better title, but here's the thing...

I was studying Elastic Net regularization and I found this function: Elastic net loss function

Sorry about the image, but it's much easier this way. So I found this as being the loss function. However, there is no lambda 1, nor lambda 2 in Scikit-Learn. Instead, we find alpha and l1_ration. After studying more, I found that Scikit-Learn alpha is actually lambda and Scikit-Learn l1_ratio is actually alpha, both from this other equation:

elastic net alternative loss function

So I guess the biggest question is: how do I go from the first equation to the last equation? How are these two equations connected?

I couldn't find a way to go from the first equation to the second one.

Again, sorry about the image, I know this isn't the best practice, but it was so much easier to just add them in here.

I couldn't find a better title, but here's the thing...

I was studying Elastic Net regularization and I found this function: Elastic net loss function

Sorry about the image, but it's much easier this way. So I found this as being the loss function. However, there is no lambda 1, nor lambda 2 in Scikit-Learn. Instead, we find alpha and l1_ration. After studying more, I found that Scikit-Learn alpha is actually lambda and Scikit-Learn l1_ratio is actually alpha, both from this other equation:

elastic net alternative loss function

So I guess the biggest question is: how do I go from the first equation to the last equation? How are these two equations connected?

I couldn't find a way to go from the first equation to the second one.

Again, sorry about the image, I know this isn't the best practice, but it was so much easier to just add them in here.

NOTE: Please consider w and beta as the same thing, the coefficients from the regression.

Source Link
Yuxxxxxx
  • 327
  • 2
  • 6
Loading