I have a linear regression model $y_a = \theta_a^T\tilde{f}$, where $\theta_a$ is a vector of learned feature weights and $\tilde{f}$ is my standardised feature vector;

$$ \tilde{f} = \frac{f - \mu}{\sigma} $$

I want to compare the learned weights $\theta_a$ with another model $y_b = \theta_b^Tf$ that was trained using the raw features $f$.

How can I convert $\theta_a$ to the appropriate units to compare with the raw-feature model weights $\theta_b$?


Some simple algebra should do the trick. The following formulation is for the two-variable equation:

$$ \hat{y} = \hat{\alpha}_\alpha + \hat{\theta}_{\alpha,1}\tilde{f_1} + \hat{\theta}_{\alpha,2}\tilde{f_2} $$ $$ \hat{y} = \hat{\alpha}_\alpha + \hat{\theta}_{\alpha,1}\frac{f_1-\mu_1}{\sigma_1} + \hat{\theta}_{\alpha,2}\frac{f_2-\mu_2}{\sigma_2} $$

Expanding terms yields an expression with the weights on the raw features $f_1$ and $f_2$:

$$\hat{y} = \hat{\alpha}_\alpha + \frac{\hat{\theta}_{\alpha,1}}{\sigma_1}f_1 -\frac{\hat{\theta}_{\alpha,1}}{\sigma_1}\mu_1 + \frac{\hat{\theta}_{\alpha,2}}{\sigma_2}f_2 -\frac{\hat{\theta}_{\alpha,2}}{\sigma_2}\mu_2 $$

To make it more clear you can define the following relationships:

$$\hat{\alpha}_b = \hat{\alpha}_\alpha - \frac{\hat{\theta}_{\alpha,1}}{\sigma_1}\mu_1 - \frac{\hat{\theta}_{\alpha,2}}{\sigma_2}\mu_2$$ $$\hat{\theta}_{b,1} = \frac{\hat{\theta}_{\alpha,1}}{\sigma_1}$$ $$\hat{\theta}_{b,2} = \frac{\hat{\theta}_{\alpha,2}}{\sigma_2}$$

  • $\begingroup$ Thanks @kanimbla :) So all I needed to do was divide by the $\sigma$ vector! >< $\endgroup$ – aaronsnoswell Mar 28 at 7:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.