What can't be expressed as a linear model? Say I have outcome variable $Y_i$ and predictors $X_{i1}$ and $X_{i2}$ for some data point $i$. Wikipedia says that a model is linear when:

the mean of the response variable is a linear combination of the parameters (regression coefficients) and the predictor variables.

I thought this meant that a model can be no more complicated than: $Y_i = \beta_1 X_{i1} + \beta_2 X_{i2}$. However, upon further reading, I found out you could handle non-linear "interactions" of the predictors like in $Y_i = \beta_1 X_{i1} + \beta_2 X_{i2} + \beta_3 X_{i1}\ X_{i2}$ by viewing $X_{i1}\ X_{i2}$ as just another predictor (which happens to be dependent on $X_{i1}$ and $X_{i2}$).  This seems to mean that you can use any (linear or non-linear) function of the predictors like $\log(X_{i1} / X_{i2}^2)$ or whatever.  Conceptually, this type of "recoding" seems like it should work for the coefficients as well.  
So: What exactly are the limits of linear regression, given you can do this kind of manipulation? 
 A: The parameter needs to enter linearly into the equation. So something like $E(Y)=\beta_1 \cos(\beta_2 x_i + \beta_3)$ would not qualify. But you can take functions of the independent variables as follows:
$E(Y)=\beta_0 + \beta_1X_i + \beta_2X^2 + \beta_3 e^{X_i}$
for example.
So the limits of linear regressions are: the mean of the $Y$ values is of the form parameter times (independent variable stuff) + parameter times (more independent variable stuff) ... and so on.
A: (Almost) Everything can be expressed as a linear model, if you don't restrict it to a finite number of parameters.
This is the basis of functional analysis and kernel regression (as in SVMs with kernels).  For instance, Fourier series - you can produce an infinite sine/cosine series, where the amplitude of the wave of each frequency gets a learned coefficient, and you can learn (almost) any function (any function whose square is integrable - which is a very weak condition).
Kernel machines, and functional analysis, are a wonderful idea, and make the world seem very beautiful - virtually everything is linear!
See http://en.wikipedia.org/wiki/Kernel_methods
The classic statistical probabilistic reference is Grace Wahba's Spline Models for Observational Data.
