4
$\begingroup$

I have two random variables $X$ and $Y$ and I'm trying to model $\mathbb{E}[Y|X]$.

To this end, I'd like to pick a collection of functions $f_1, f_2 \dots f_n : \mathbb{R} \to \mathbb{R}$ and then fit a model to my data set $(x_i, y_i)_{1 \dots m}$ by OLS:

$$\mathrm{argmin}_{\alpha_1 \dots \alpha_n} \sum_{i=1}^m \left(y_i - \sum_{j=1}^n \alpha_j f_j(x_i) \right)^2$$

I'm looking for a reference that compares different choices of families of $\{ f_i\}$ and answers questions like:

  • Will any linearly independent $f_i$ work?
  • Is an orthogonal collection (Laguerre polynomials, Fourier basis) better and mathematically, why?

I am particularly keen to learn about the numerical stability of the resulting least squares problem. Do some bases $\{ f_i \}$ work fair noticeably worse if the least squares problem is solved by Cholesky or QR factorisation instead of SVD?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.