# Choosing the basis functions in a linear regression

I have two random variables $X$ and $Y$ and I'm trying to model $\mathbb{E}[Y|X]$.

To this end, I'd like to pick a collection of functions $f_1, f_2 \dots f_n : \mathbb{R} \to \mathbb{R}$ and then fit a model to my data set $(x_i, y_i)_{1 \dots m}$ by OLS:

$$\mathrm{argmin}_{\alpha_1 \dots \alpha_n} \sum_{i=1}^m \left(y_i - \sum_{j=1}^n \alpha_j f_j(x_i) \right)^2$$

I'm looking for a reference that compares different choices of families of $\{ f_i\}$ and answers questions like:

• Will any linearly independent $f_i$ work?
• Is an orthogonal collection (Laguerre polynomials, Fourier basis) better and mathematically, why?

I am particularly keen to learn about the numerical stability of the resulting least squares problem. Do some bases $\{ f_i \}$ work fair noticeably worse if the least squares problem is solved by Cholesky or QR factorisation instead of SVD?