# Un-standardize feature weights

I have a linear regression model $$y_a = \theta_a^T\tilde{f}$$, where $$\theta_a$$ is a vector of learned feature weights and $$\tilde{f}$$ is my standardised feature vector;

$$\tilde{f} = \frac{f - \mu}{\sigma}$$

I want to compare the learned weights $$\theta_a$$ with another model $$y_b = \theta_b^Tf$$ that was trained using the raw features $$f$$.

How can I convert $$\theta_a$$ to the appropriate units to compare with the raw-feature model weights $$\theta_b$$?

Some simple algebra should do the trick. The following formulation is for the two-variable equation:

$$\hat{y} = \hat{\alpha}_\alpha + \hat{\theta}_{\alpha,1}\tilde{f_1} + \hat{\theta}_{\alpha,2}\tilde{f_2}$$ $$\hat{y} = \hat{\alpha}_\alpha + \hat{\theta}_{\alpha,1}\frac{f_1-\mu_1}{\sigma_1} + \hat{\theta}_{\alpha,2}\frac{f_2-\mu_2}{\sigma_2}$$

Expanding terms yields an expression with the weights on the raw features $$f_1$$ and $$f_2$$:

$$\hat{y} = \hat{\alpha}_\alpha + \frac{\hat{\theta}_{\alpha,1}}{\sigma_1}f_1 -\frac{\hat{\theta}_{\alpha,1}}{\sigma_1}\mu_1 + \frac{\hat{\theta}_{\alpha,2}}{\sigma_2}f_2 -\frac{\hat{\theta}_{\alpha,2}}{\sigma_2}\mu_2$$

To make it more clear you can define the following relationships:

$$\hat{\alpha}_b = \hat{\alpha}_\alpha - \frac{\hat{\theta}_{\alpha,1}}{\sigma_1}\mu_1 - \frac{\hat{\theta}_{\alpha,2}}{\sigma_2}\mu_2$$ $$\hat{\theta}_{b,1} = \frac{\hat{\theta}_{\alpha,1}}{\sigma_1}$$ $$\hat{\theta}_{b,2} = \frac{\hat{\theta}_{\alpha,2}}{\sigma_2}$$

• Thanks @kanimbla :) So all I needed to do was divide by the $\sigma$ vector! >< – aaronsnoswell Mar 28 at 7:58