1
$\begingroup$

I am learning multiple linear regression. I know for simple linear regression if I have the following X and Y values, then how to predict Y'. Like if I have the following datas:

X     Y   Y'
1     0   ?
0     1   ?
0     1   ?
0     0   ?

then I can calculate a and b and get Y' from the equation y = a + bx as shown here http://www.easycalculation.com/statistics/learn-regression.php. Now, I have multiple independent variables then what is the formula to calculate the Y' using multiple linear regression? My current datas are:

X1   X2   X3   X4   X5   Y  Y'
 1    0    0    1    0    1  ?
 0    1    0    1    0    0  ?
 0    0    0    0    1    0  ?
 1    0    1    0    0    1  ?

Can anybody show me the formula to predict Y'? I have searched a lot but those shows me scatter plots and analysis and all. I just want to get the Y' values. I didn't get any tutorial like I got for linear regression (link provided above) where it is shown clearly how to calculate Y' values.

Edit: So, according to the answers my X and Y matrix are:

 X = 1   1    0   0    1    0    
     1   0    1   0    1    0    
     1   0    0   0    0    1   
     1   1    0   1    0    0  

 Y = 1
     0
     0
     1

and then I calculate betas with help of (X'X)^-1 X'Y equation. Then put the x1, x2, x3, values to predict y1, y2, y3 and so on - right?

Improve this question
$\endgroup$

2 Answers 2

Reset to default
0
$\begingroup$

Maybe you are searching with the wrong keywords. Consider multivariate linear regression and take a look at wiki and this other example for a start.

Improve this answer
$\endgroup$
5
  • $\begingroup$ @pedrofigueria I have seen this wiki before. I know the equation "y = a + b1x1 + b2x2 + b3x3 + b4x4 + .....". In wiki, in the place where it's written, the least square estimates, ^b, can now be obtained: why has the first column of X matrix been taken 1? I can understand the other values corresponds to x and y. $\endgroup$
    – user2567857
    Commented May 23, 2014 at 12:40
  • $\begingroup$ the lower scripts 1,2,3, ... n correspond to the independent variables and their associated regression coefficients. The matrix $\beta$ is just a way of representing all these in a matrix such that each line correspond to a $\beta_i$. But I might have misunderstood your question. $\endgroup$
    – pedrofigueira
    Commented May 23, 2014 at 12:46
  • $\begingroup$ Yes, you misunderstood. I am taking of X matrix. It's writen X = (1 41.9 29.1) and below there are more rows. I can't show the full X matrix here. Why the first columns of this X matrix is 1? Like the other column values which are 41.9 and 29.1 represent X values of the two independent variable. $\endgroup$
    – user2567857
    Commented May 23, 2014 at 12:53
  • $\begingroup$ Now I realize you are talking of the second link and table on the example. Make sure you write what you mean explicitly. The i is just a running index on the observation, i.e. the number of rows. The $x_{i,1}$ is what we called $x_1$, and $x_{i,2}$ is $x_2$. $\endgroup$
    – pedrofigueira
    Commented May 23, 2014 at 13:00
  • $\begingroup$ Then why it is always 1 and why the values are not like 1 2 3 ....17for that example? $\endgroup$
    – user2567857
    Commented May 23, 2014 at 14:08
1
$\begingroup$

If by "multiple independent variables", you mean that all the X are pairwise uncorrelated (in the sample), then you're lucky. The formula to predict Y is just the sum of the betas of the simple linear regressions.

If not, the formula is too complicated to state in regular algebra. In linear algebra, the vector of the beta parameters is (X'X)^-1 X'Y. You need to learn some linear algebra to understand the mechanics of multivariate regression: http://en.wikipedia.org/wiki/Linear_least_squares_(mathematics)

To respond to your question in the comments, the first column of X is all 1s because that is the intercept. For a simple linear regression, too, you can think of the first variable as a vector with all 1s: y = a + bx <=> y=a*1+bx.

Improve this answer
$\endgroup$
6
  • $\begingroup$ So, why do I need to take that column having all the values as 1? What if I leave that column and simply put the x values in my X matrix? $\endgroup$
    – user2567857
    Commented May 23, 2014 at 14:12
  • $\begingroup$ Doing so implicitly sets a, the intercept, to be 0. For y=a+bx, that forces the line to go through the origin and will also change b. It's the same in higher dimensions. It's almost always a bad idea to omit the intercept. $\endgroup$
    – CloseToC
    Commented May 23, 2014 at 14:17
  • $\begingroup$ Please see my edited answer, where I have edited of what I understood from the answers, as none of the answers tells me explicitly the values of X and Y. Did I understood right that all the first column of X matrix will be 1? $\endgroup$
    – user2567857
    Commented May 23, 2014 at 14:20
  • $\begingroup$ Yes. If you have n observations, all n rows of the first column of X are 1. To predict the first observation, y_1, you calculate a+b1*x_1+b2*x_2 etc.. a, b1, b2 etc is what the vector (X'X)^-1 X'Y gives you. The first row in that vector is a, the second b1, and so on $\endgroup$
    – CloseToC
    Commented May 23, 2014 at 14:26
  • 1
    $\begingroup$ You should not worry to much about that column of ones. That is just one way of representing the intercept. In python's scikit linear regression (scikit-learn.org/stable/modules/generated/…) you never specify that column, i.e., you never provide it as an input. I would advise you to see how to input the data on a given regression algorithm once you understood the principles of the method. You might find it easier that you thought. $\endgroup$
    – pedrofigueira
    Commented May 23, 2014 at 14:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.