2
$\begingroup$

My question regards the linear model $Y = X \cdot \beta + \varepsilon$. I currently attend a lecture on linear models and I realize this issue is very basic. In our script there are plenty of equations that describe the properties of that specific linear model including it's solution $\hat{\beta} = (X^T \cdot X)^{-1} \cdot X^T \cdot Y$. My problem lies in understanding the concept.

My understanding is this: We have $n$ observations of some phenomenon, where each observation includes a part we have precise or known data for ($X$ or $\beta$) and then the observed variables we try to estimate $Y$. We try to understand the influence of $p$ factors which is why $X$ has dimension $n \times p$ and $\beta$ has dimension p.

Is my understanding so far correct? What do the indiviual variables represent? What is the meaning of $\hat{\beta}$? It is an estimator but what does it estimate and why do we care?

Edit: I guess I was naive with my question. The problem is that someone who has scarce knowledge of a topic has a hard time using proper terminology when describing their point of not understanding. Those questions were not asked to get answered one by one but more to express that I don't understand the intuition of the model. I was hoping someone who does understand it well can summarize the idea in an intuitive way.

Improve this question
$\endgroup$
6
  • $\begingroup$ Would you be solving for the data?? $\endgroup$
    – whuber ♦
    Mar 25, 2021 at 16:30
  • 1
    $\begingroup$ Why are you making that comment? It serves no purpose apart from insulting my intelligence. If you don't want to help me understand please just spare me. $\endgroup$
    – lpnorm
    Mar 25, 2021 at 16:37
  • 1
    $\begingroup$ No insult intended. I'm just trying to understand what kind of responses, and at what level of sophistication, you might be seeking, and I found it rather baffling that answers to some of your questions appear to be right at the surface of your question. That led me to suppose that perhaps you haven't formulated the question in quite the way you intended. It might also be worth pointing out that this site has a huge number of posts about these issues and you might find it faster and more rewarding to research some of them. $\endgroup$
    – whuber ♦
    Mar 25, 2021 at 17:04
  • 3
    $\begingroup$ Then no offense taken. My level is that of a mathematized lecture where we use methods of linear algebra to proof several properties of this setup but while I understand each step we take I have no idea what it actually is that we are doing. There is no explanation in my script as to what exactly this model describes and when it is used. I was looking for confirmation of my assumptions and maybe a short explanation of what $\beta$ is. $\endgroup$
    – lpnorm
    Mar 25, 2021 at 17:17
  • 1
    $\begingroup$ Searching for solved examples of regression in practice should be helpful. This looks like a great introductory resource, and uses the same notation as OP: genomicsclass.github.io/book/pages/intro_using_regression.html Maybe check that one out and see if it helps narrow down the questions? $\endgroup$
    – juod
    Mar 26, 2021 at 1:08

1 Answer 1

Reset to default
3
$\begingroup$

Here is why further prompts on your question have been made by commenters. It is to avoid the following vague answer, which I suspect is not what you want:

Which is the 'known data', $X$ or $\beta$?

The 'known data' is $X$.

Then what is $\beta$? Is there something random about $X$ or $\beta$?

$\beta$ is a fixed unknown parameter, so is not random. $X$ consists of $n$ observations of a random vector $\mathbf{x} = (X_1, ..., Xp)$, so can be viewed as random.

What is the solution $\hat{\beta}$?

It is an estimator which satisfies certain desirable properties, and which is used to estimate the fixed, unknown parameter $\beta$. $\hat{\beta}$ is a function of the data, which you have specified in your formula. Under appropriate assumptions about $\varepsilon$, estimating $\beta$ means that you are estimating a conditional distribution of $Y$ given $X$, as specified by your model.

Improve this answer
$\endgroup$
4
  • 1
    $\begingroup$ “ 𝛽 is a fixed unknown parameter”—unless you’re a Bayesian, in which case it’s also governed by a random variable. Overall, this is a very good, direct answer. $\endgroup$
    – Arya McCarthy
    Mar 25, 2021 at 17:53
  • 1
    $\begingroup$ @AryaMcCarthy. Thank you for pointing out that out. Yes I made the guess that OP did not have a Bayesian setting in mind, and so felt including it would perhaps confuse the issue somewhat further. $\endgroup$
    – microhaus
    Mar 25, 2021 at 18:00
  • $\begingroup$ Thank you for trying to answer what apparantly is an ill posed question. In our script we often have $E[Y] = E[X \cdot \beta] +E[\varepsilon]$ with $E[\varepsilon] = 0$ and $E[X \cdot \beta] = X \cdot \beta$ which suggests that X is not random. If X is random, is $\varepsilon$ supposed to contain that randomness as well? Could you update your answer? Also I would love to hear about the Bayesian approach as well if explained in an intuitive fashion. $\endgroup$
    – lpnorm
    Mar 26, 2021 at 11:07
  • $\begingroup$ I think a tutorial on the Bayesian perspective is too much to ask microhaus to edit in, because this will quickly turn into the length of a book chapter. Keep your question focused. You can ask about Bayesian linear regression in another question. $\endgroup$
    – Arya McCarthy
    Mar 26, 2021 at 12:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.