# Variable Selection of Linear Regression via Simulated Annealing

I am dealing a problem currently about the simulated annealing.

The problem is:

$$Y = β_1X_1 + β_2X_2+ \dots + β_{1000}X_{1000} + \epsilon,\ \epsilon ∼ \mathcal{N}(0, σ_2)$$ We take the Residual Sum of Squares (RSS) as the loss function in this linear regression problem with simulated annealing. Please ﬁnd the best linear model with only three explanatory variables. In other words, only three out of $$β_1, \dots, β_{1000}$$ are non-zero.

I know the details of simulated annealing and know how to implement it. But what do best linear model and explanatory variable mean? I try to choose three maximum values of $$\beta$$ each time but the final result differs each time. So what is the criterion of best? Is there anything I am missing? Thanks in advance!

• Could you please explain what the operation "$+\gt$" is intended to mean? BTW, your question is directly answered in the quotation: "best" means a model with the lowest RSS value (among those with at most three nonzero coefficients).
– whuber
Dec 19, 2020 at 13:48
• @whuber I am sorry that I did add an additional "$>$" in the formulation. But each time the $3$ components with lowest RSS values are different. Dec 20, 2020 at 7:15
• Could you explain what you mean by "each time"? You seem to be referring to some kind of repeated process. Is it possible you are trying to ask this question?
– whuber
Dec 20, 2020 at 15:37
• @whuber I use 0 as initialization stage. And choose three maximum values of $\beta$ in each iteration of the simulated annealing until convergence. But when I run simulated annealing algorithms for several times, I get different answer each time. Dec 22, 2020 at 0:48
• @whuber Thanks for your reply. I understand your point now. Thank you very much! Dec 23, 2020 at 1:03