Return to Question

Disclaimer: this is not homework, just preparation for an exam I'll be taking

How do I incorporate an offset term into a software library that performs least squares (without offset)?

Let me give a simple example which I'd like for the answerer to base their answer on:

Consider a dataset $(x_1,y_1),...,(x_m,y_m)\in R \times R$, i.e., both $x_i$ and $y_i$ are real-valued scalars for all $i$. Suppose we wish to fit a linear+offset model, $$\hat{y} = ax + b$$ Suppose we have a software library that performs least squares (without offset). Explain how we can use that software library to fit the linear+offset model.

Now, by my understanding, we have that the software library performs linear regression on $X\in R^m$ and $y\in R^m$. In order to incorporate an offset, we should add a column of "ones" such that $X$ is substituted by $(X,\mathbf{1})$ where $\mathbf{1}\in R^m$ is a vector with each entry equal to $1$. We also should substitute $a\in R$ by $(a,b)$ where $b\in R$. Now, without the offset, the normal equations for computing $a$ was $$a= (X^TX)^{-1}X^Ty$$ and with the above substitutions the software library calculates $$(X^TX)a + X^T\mathbf{1}b = X^Ty\\ \mathbf{1}^TXa + mb = \mathbf{1}^Ty$$

Disclaimer: this is not homework, just preparation for an exam I'll be taking

How do I incorporate an offset term into a software library that performs least squares (without offset)?

Let me give a simple example which I'd like for the answerer to base their answer on:

Consider a dataset $(x_1,y_1),...,(x_m,y_m)\in R \times R$, i.e., both $x_i$ and $y_i$ are real-valued scalars for all $i$. Suppose we wish to fit a linear+offset model, $$\hat{y} = ax + b$$ Suppose we have a software library that performs least squares (without offset). Explain how we can use that software library to fit the linear+offset model.

Now, by my understanding, we have that the software library performs linear regression on $X\in R^m$ and $y\in R^m$. In order to incorporate an offset, we should add a column of "ones" such that $X$ is substituted by $(X,\mathbf{1})$ where $\mathbf{1}\in R^m$ is a vector with each entry equal to $1$. We also should substitute $a\in R$ by $(a,b)$ where $b\in R$. Now, without the offset, the normal equations for computing $a$ was $$a= (X^TX)^{-1}X^Ty$$ and with the above substitutions the software library calculates $$(X^TX)a + X^T\mathbf{1}b = X^Ty\\ \mathbf{1}^TXa + mb = \mathbf{1}^Ty$$

Is my understanding correct?

How do I incorporate an offset term into a software library that performs least squares (without offset)?

Let me give a simple example which I'd like for the answerer to base their answer on:

Consider a dataset $(x_1,y_1),...,(x_m,y_m)\in R \times R$, i.e., both $x_i$ and $y_i$ are real-valued scalars for all $i$. Suppose we wish to fit a linear+offset model, $$\hat{y} = ax + b$$ Suppose we have a software library that performs least squares (without offset). Explain how we can use that software library to fit the linear+offset model.

Now, by my understanding, we have that the software library performs linear regression on $X\in R^m$ and $y\in R^m$. In order to incorporate an offset, we should add a column of "ones" such that $X$ is substituted by $(X,\mathbf{1})$ where $\mathbf{1}\in R^m$ is a vector with each entry equal to $1$. We also should substitute $a\in R$ by $(a,b)$ where $b\in R$. Now, without the offset, the normal equations for computing $a$ was $$a= (X^TX)^{-1}X^Ty$$ and with the above substitutions the software library calculates $$(X^TX)a + X^T\mathbf{1}b = X^Ty\\ \mathbf{1}^TXa + mb = \mathbf{1}^Ty$$

Disclaimer: this is not homework, just preparation for an exam I'll be taking

How do I incorporate an offset term into a software library that performs least squares (without offset)?

Let me give a simple example which I'd like for the answerer to base their answer on:

Consider a dataset $(x_1,y_1),...,(x_m,y_m)\in R \times R$, i.e., both $x_i$ and $y_i$ are real-valued scalars for all $i$. Suppose we wish to fit a linear+offset model, $$\hat{y} = ax + b$$ Suppose we have a software library that performs least squares (without offset). Explain how we can use that software library to fit the linear+offset model.

Now, by my understanding, we have that the software library performs linear regression on $X\in R^m$ and $y\in R^m$. In order to incorporate an offset, we should add a column of "ones" such that $X$ is substituted by $(X,\mathbf{1})$ where $\mathbf{1}\in R^m$ is a vector with each entry equal to $1$. We also should substitute $a\in R$ by $(a,b)$ where $b\in R$. Now, without the offset, the normal equations for computing $a$ was $$a= (X^TX)^{-1}X^Ty$$ and with the above substitutions the software library calculates $$(X^TX)a + X^T\mathbf{1}b = X^Ty\\ \mathbf{1}^TXa + mb = \mathbf{1}^Ty$$