Linked Questions
27 questions linked to/from What are the differences between stochastic and fixed regressors in linear regression model?
4
votes
2
answers
496
views
Regression with random X [duplicate]
Suppose we have a standard regression model
$$Y= X\beta + \epsilon$$
with
$$\epsilon \sim \sigma^2$$
$$X \sim N(\mu,\gamma^2)$$
Are the estimated coefficients the same as if $X$ was fixed?
Is ...
- 523
1
vote
1
answer
365
views
Random Explanatory / Independent Variables [duplicate]
Are explanatory variables in regression always considered non-stochastic? If the explanatory variables are random or stochastic will the regression be still valid? What are the implications on the ...
- 587
2
votes
1
answer
179
views
Meaning of & intuition behind predictors being fixed in linear regression [duplicate]
My question is a bit naive. I'm trying to get the exact & clear meaning of the phrase "predictor variables are fixed and not random in linear regression".
According to my understanding, ...
2
votes
0
answers
29
views
Regression: What is the difference between assuming the covariates are random or not random? [duplicate]
I often see regression expressed in two ways.
The covariates are random: In this scenario, we have $(x_i,y_i) \sim G$ for some distribution $G$ and are i.i.d. for $i = 1, \cdots, n$. We then posit $...
39
votes
6
answers
9k
views
Under which assumptions a regression can be interpreted causally?
First, don't panic. Yes, there are many similar question on this site. But I believe none gives a conclusive answer to the question below. Please bear with me.
Consider a data generation process $\...
- 2,757
20
votes
5
answers
3k
views
Definition and delimitation of regression model
An embarrassingly simple question -- but it seems it has not been asked on Cross Validated before:
What is the definition of a regression model?
Also a support question,
What is not a regression ...
- 63.6k
21
votes
2
answers
6k
views
What is the difference between conditioning on regressors vs. treating them as fixed?
Sometimes we assume that regressors are fixed, i.e. they are non-stochastic. I think that means all our predictors, parameter estimates etc. are unconditional then, right? Might I even go so far that ...
- 997
7
votes
2
answers
3k
views
What does the assumption: "The independent variable is not random." in OLS mean?
What does the assumption: "The independent variable is not random." in OLS mean? How can you verify that hypothesis?
- 1,035
7
votes
3
answers
561
views
Which likelihood function is used in linear regression?
When trying to derive the maximum likelihood estimation for a linear regression, We start by a likelihood function. Does it matter if we use either of these 2 forms?
$P(y|x,w)$
$P(y,x|w)$
All pages ...
- 1,360
3
votes
2
answers
2k
views
Regression and the CEF
I recently read in this page (https://www.timlrx.com/2018/02/26/notes-on-regression-approximation-of-the-conditional-expectation-function/#fn1) that:
"Regression offers a way of approximating ...
1
vote
2
answers
963
views
Regression's population parameters
Suppose I've specified a linear regression model:
$$
Y = \beta_0 + \beta_1 X + \epsilon
$$
where $\beta_0$, $\beta_1$ are the population parameters. My question is: why are these parameters ...
- 137
5
votes
1
answer
1k
views
Proof that Regression Sum of Squares and Residual Sum of Squares are independent random variables
Having consulted a number of sources, I still can't find a complete proof that Regression Sum of Squares ($SS_{regression}$) and ($SS_{residual}$) are independent random variables. I'll be doubly ...
- 4,156
2
votes
2
answers
808
views
Does the OLS estimator in simple linear regression converge a.s.?
Consider the following model.
Assume $(x_i, u_i)$ is sequence of independent identically distributed random vectors in $\mathbf{R}^{d+1}:$
$x_i$ are $\mathbf{R}^d$-value random vectors, which will ...
- 402
2
votes
0
answers
2k
views
Do fixed design prediction/estimation error guarantees translate to random design for linear regression? When and How? [closed]
Suppose I have an independent vector $X$ and a dependent scalar random variable $Y$ and I wish to construct a regression model to predict $Y$ using $X$ given data $\{(x_i,y_i)\}_{i=1}^{n}$. For ...
4
votes
1
answer
395
views
Why does regression model theory not use measure-theoretic sigma-field type notation but counting process models do?
I have been studying counting process theory for time to recurrent event processes and am interested in the explicit use of the conditioning set in the model notation;
$$E[dN(t)|\mathcal{F}_{t^{-}}]=\...
- 698
5
votes
3
answers
174
views
Foundations behind Linear Regression / Statistical Modelling
I've always struggled with the foundations behind the concept of modelling (and specifically regression) - what is random, what is not, what we are modelling.
I think I have a grasp of it - but I'd ...
- 569
2
votes
1
answer
408
views
non stochastic regressors and causation
Randomized controlled experiment is base case for causality (also) in regression.
However currently I’m analyzing the role of causality in linear regression as shown in many econometrics textbook. ...
- 4,896
2
votes
1
answer
241
views
Clarification on the assumptions $E[u|x]=0$ and the $x_i$ being fixed in repeated samples in Wooldridge Introductory Econometrics
The author is writing on the assumption $E[u|x]=0$.
The part of the text which is not clear to me is this (the red lines emphasize where the critical portions are located) :
In the first piece I don'...
- 356
3
votes
1
answer
391
views
Why is each observation in a sample considered a random variable in linear regression?
I have the following excerpt in my statistics textbook:
I am confused by the sentence: "Another way statisticians treat this model is that, assume $X_1...X_n$ are random variables, we make ...
- 641
2
votes
1
answer
162
views
Confused with the fundamental assumptions of Frequentist and Bayesian Linear Regression
In Frequentist Linear Regression, I have seen 2 approaches which lead to basically similar models. We have $W,y,X,\epsilon$ related as $y=W^TX+\epsilon$, where $y$ is the dependent random variable, ...
- 133
2
votes
1
answer
115
views
Deriving the posterior distribution over the model parameters: are the model parameters and data independent?
We are told (in Section 9.2.3, Deisenroth et al.: Mathematics for Machine Learning) that we can compute the posterior over a model's parameters $\boldsymbol\theta$ (here in the context of linear ...
1
vote
1
answer
225
views
In least square linear regression model, why does the test t-statistic of $\hat{\beta}$ follow a t distribution?
In the least square linear regression model, if the explanatory variables and the error term are independent, and the error term is normal, why does the t-statistic of $\hat{\beta}$ follow a t ...
0
votes
0
answers
208
views
Non-stochastic vs Stochastic regressors and sampling distributions and causation?
I was wondering if I understand these correctly. Would an example of a stochastic regressor be weather? so when thinking about the sampling distribtuion and causality, I would think of repeated ...
- 611
1
vote
2
answers
119
views
Why is the likelihood function for linear regression based on $𝑝(𝑦|𝑥)$ instead of $𝑝(𝑥,𝑦)$
Likelihood functions typically based on the joint probabilities of the variables involved. In linear regression we have variables 𝑥 and 𝑦, but derivations for MLE under a zero meaned gaussian ...
- 595
2
votes
1
answer
62
views
Notation of the Likelihood Term in Bayesian Neural Networks
I see that in Bayesian neural networks likelihood function is defined in two ways:
$p(W|D) = Z^{-1} p(D|W)p(W)$
or
$p(W|y,x)=Z^{-1}p(y|x,W)p(W)$
Are there a slight difference in interpreting $p(D|W)$ ...
- 177
0
votes
1
answer
49
views
Do we assume that the regressors are uncorrelated with the unobserved error $\epsilon$ for least squares?
I recall seeing sources in the past state that the Gauss-Markov assumptions assume that the regressors are uncorrelated with $\epsilon$ in order to make $E[\hat{\beta}] = \beta$. But is this ...
- 414
0
votes
0
answers
29
views
Does maximum likelihood need to account for the probability of input covariates?
I am reading "Probabilistic Machine Learning" by K. Murphy. In it, he defines the likelihood of a dataset as
However, this dataset $D$ is defined as:
So if all $x_n, y_n$ are random ...
- 11