1
$\begingroup$

One way of making linear regression applicable more widely is to use basis expansions, i.e., adding more features to the input set. Suppose that the data is described by a p-tuple, $(x_1 , x_2 , . . . , x_p )$. Comment on the utility of the following sets of features. Specifically describe the family of functions that can be represented by a linear combination of these features.

$(a)( x_1 , . . . , x_p , x_1^2 , x_1 x_2 , x_1 x_3 , . . . , x_1 x_p , x_ 2^2 , x_2 x_3 , . . . , x^2_p)$
$ (b) (x^2_1 , x^2_2 , . . . , x^2_p)$

How to solve this type of questions? Any hint or idea.

My Attempt : I have studied linear regression from Stanford notes Now according to this the equation of the predicted value of $y$ is given by a linear equation of the feature variables, but here in this question the feature variables given are not linear so will it be of the same form?

Further by family of function does it imply finding the equation of $y$ or it has some other meaning? Here by $y$ I mean value being predicted using linear regression.

Improve this question
$\endgroup$
1
  • $\begingroup$ Please add the [self-study] tag & read its wiki. Then tell us what you understand thus far, what you've tried & where you're stuck. We'll provide hints to help you get unstuck. $\endgroup$
    – gung - Reinstate Monica
    Commented Apr 25, 2018 at 20:42

2 Answers 2

Reset to default
1
$\begingroup$

I hope I understood your question. The equation for linear regression (without the intercept) can be written as follows: \begin{equation} y_i=\beta_1x_{i,1}+\beta_2x_{i,2}+…+\beta_{p}x_{i,p}+\epsilon_i. \end{equation} For $(a)$ using the transformation $x_i=x_i+(x_i * x_{i+1})$ and we get the following: \begin{equation} y_i=\beta_1x_{i,1}+\beta_1x^2_{i,1}+\beta_2x_{i,2}+\beta_2x_{i,2}+\beta_2x_{i,2}x_{i,3}+...+\beta_{p}x^2_{i,p}+\epsilon_i \end{equation} For $(b)$ using the transformation $x_i=x_i^2$ and we get the following: \begin{equation} y_i=\beta_1x^2_{i,1}+\beta_2x^2_{i,2}+…+\beta_{p}x^2_{i,p}+\epsilon_i. \end{equation} where $i=1,2,...,t$ and $t$ is some finite integer.

Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ By "family of functions" surely the question means functions of the $x_i$ rather than of the parameters. Each of the two families has a simple succinct geometric description--but neither is linear. $\endgroup$
    – whuber ♦
    Commented Apr 25, 2018 at 21:29
  • $\begingroup$ I did not understand what you mean here. I meant that (a) and (b) are still linear combination as asked in the question. I have removed that line, maybe it was causing some confusion. $\endgroup$
    – Waqas
    Commented Apr 26, 2018 at 7:19
0
$\begingroup$

Answer to your first question:

Linear equation means linear combination of the features/variables. In linear equation we focus on the fact that the combination of features must be linear. Here we does not focus on the form of the specific variable. So, irrespective of what form any variable takes, whether it is polynomial or exponential, if these variables are linearly combined then it is linear equation.

Answer to your second question:

Yes, finding the family of functions means findings all the functions for which linear combination of all the variables gives the value of y.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.