Linear Regression feature transformation One way of making linear regression applicable more widely is to use basis
expansions, i.e., adding more features to the input set. Suppose that the
data is described by a p-tuple, $(x_1 , x_2 , . . . , x_p )$. Comment on the utility of
the following sets of features. Specifically describe the family of functions
that can be represented by a linear combination of these features.
$(a)( x_1 , . . . , x_p , x_1^2 , x_1 x_2 , x_1 x_3 , . . . , x_1 x_p , x_ 2^2 , x_2 x_3 , . . . , x^2_p)$$
(b) (x^2_1 , x^2_2 , . . . , x^2_p)$
How to solve this type of questions? Any hint or idea.
My Attempt :
I have studied linear regression from Stanford notes
Now according to this the equation of the predicted value of $y$ is given by a linear equation of the feature variables, but here in this question the feature variables given are not linear so will it be of the same form?
Further by family of function does it imply finding the equation of $y$ or it has some other meaning?
Here by $y$ I mean value being predicted using linear regression.
 A: I hope I understood your question. The equation for linear regression (without the intercept) can be written as follows:
\begin{equation}
y_i=\beta_1x_{i,1}+\beta_2x_{i,2}+…+\beta_{p}x_{i,p}+\epsilon_i.
\end{equation}
For $(a)$ using the transformation $x_i=x_i+(x_i * x_{i+1})$ and we get the following:
\begin{equation}
y_i=\beta_1x_{i,1}+\beta_1x^2_{i,1}+\beta_2x_{i,2}+\beta_2x_{i,2}+\beta_2x_{i,2}x_{i,3}+...+\beta_{p}x^2_{i,p}+\epsilon_i
\end{equation}
For $(b)$ using the transformation $x_i=x_i^2$ and we get the following:
\begin{equation}
y_i=\beta_1x^2_{i,1}+\beta_2x^2_{i,2}+…+\beta_{p}x^2_{i,p}+\epsilon_i.
\end{equation}
where $i=1,2,...,t$ and $t$ is some finite integer.
A: Answer to your first question:
Linear equation means linear combination of the features/variables.
In linear equation we focus on the fact that the combination of features must be linear. Here we does not focus on the form of the specific variable. So, irrespective of what form any variable takes, whether it is polynomial or exponential, if these variables are linearly combined then it is linear equation.
Answer to your second question:
Yes, finding the family of functions means findings all the functions for which linear combination of all the variables gives the value of y.
