Linked Questions
27 questions linked to/from What are the differences between stochastic and fixed regressors in linear regression model?
0
votes
0
answers
29
views
Does maximum likelihood need to account for the probability of input covariates?
I am reading "Probabilistic Machine Learning" by K. Murphy. In it, he defines the likelihood of a dataset as
However, this dataset $D$ is defined as:
So if all $x_n, y_n$ are random ...
- 11
3
votes
1
answer
391
views
Why is each observation in a sample considered a random variable in linear regression?
I have the following excerpt in my statistics textbook:
I am confused by the sentence: "Another way statisticians treat this model is that, assume $X_1...X_n$ are random variables, we make ...
- 641
2
votes
1
answer
179
views
Meaning of & intuition behind predictors being fixed in linear regression [duplicate]
My question is a bit naive. I'm trying to get the exact & clear meaning of the phrase "predictor variables are fixed and not random in linear regression".
According to my understanding, ...
2
votes
2
answers
808
views
Does the OLS estimator in simple linear regression converge a.s.?
Consider the following model.
Assume $(x_i, u_i)$ is sequence of independent identically distributed random vectors in $\mathbf{R}^{d+1}:$
$x_i$ are $\mathbf{R}^d$-value random vectors, which will ...
- 402
2
votes
1
answer
62
views
Notation of the Likelihood Term in Bayesian Neural Networks
I see that in Bayesian neural networks likelihood function is defined in two ways:
$p(W|D) = Z^{-1} p(D|W)p(W)$
or
$p(W|y,x)=Z^{-1}p(y|x,W)p(W)$
Are there a slight difference in interpreting $p(D|W)$ ...
- 177
0
votes
1
answer
49
views
Do we assume that the regressors are uncorrelated with the unobserved error $\epsilon$ for least squares?
I recall seeing sources in the past state that the Gauss-Markov assumptions assume that the regressors are uncorrelated with $\epsilon$ in order to make $E[\hat{\beta}] = \beta$. But is this ...
- 414
2
votes
1
answer
241
views
Clarification on the assumptions $E[u|x]=0$ and the $x_i$ being fixed in repeated samples in Wooldridge Introductory Econometrics
The author is writing on the assumption $E[u|x]=0$.
The part of the text which is not clear to me is this (the red lines emphasize where the critical portions are located) :
In the first piece I don'...
- 356
5
votes
1
answer
1k
views
Proof that Regression Sum of Squares and Residual Sum of Squares are independent random variables
Having consulted a number of sources, I still can't find a complete proof that Regression Sum of Squares ($SS_{regression}$) and ($SS_{residual}$) are independent random variables. I'll be doubly ...
- 4,156
2
votes
0
answers
29
views
Regression: What is the difference between assuming the covariates are random or not random? [duplicate]
I often see regression expressed in two ways.
The covariates are random: In this scenario, we have $(x_i,y_i) \sim G$ for some distribution $G$ and are i.i.d. for $i = 1, \cdots, n$. We then posit $...
39
votes
6
answers
9k
views
Under which assumptions a regression can be interpreted causally?
First, don't panic. Yes, there are many similar question on this site. But I believe none gives a conclusive answer to the question below. Please bear with me.
Consider a data generation process $\...
- 2,757
2
votes
1
answer
162
views
Confused with the fundamental assumptions of Frequentist and Bayesian Linear Regression
In Frequentist Linear Regression, I have seen 2 approaches which lead to basically similar models. We have $W,y,X,\epsilon$ related as $y=W^TX+\epsilon$, where $y$ is the dependent random variable, ...
- 133
3
votes
2
answers
2k
views
Regression and the CEF
I recently read in this page (https://www.timlrx.com/2018/02/26/notes-on-regression-approximation-of-the-conditional-expectation-function/#fn1) that:
"Regression offers a way of approximating ...
1
vote
2
answers
963
views
Regression's population parameters
Suppose I've specified a linear regression model:
$$
Y = \beta_0 + \beta_1 X + \epsilon
$$
where $\beta_0$, $\beta_1$ are the population parameters. My question is: why are these parameters ...
- 137
2
votes
1
answer
115
views
Deriving the posterior distribution over the model parameters: are the model parameters and data independent?
We are told (in Section 9.2.3, Deisenroth et al.: Mathematics for Machine Learning) that we can compute the posterior over a model's parameters $\boldsymbol\theta$ (here in the context of linear ...
5
votes
3
answers
174
views
Foundations behind Linear Regression / Statistical Modelling
I've always struggled with the foundations behind the concept of modelling (and specifically regression) - what is random, what is not, what we are modelling.
I think I have a grasp of it - but I'd ...
- 569