Linked Questions

5 votes
2 answers
692 views

Regression with random X [duplicate]

Suppose we have a standard regression model $$Y= X\beta + \epsilon$$ with $$\epsilon \sim \sigma^2$$ $$X \sim N(\mu,\gamma^2)$$ Are the estimated coefficients the same as if $X$ was fixed? Is ...
user2879934's user avatar
1 vote
1 answer
453 views

Random Explanatory / Independent Variables [duplicate]

Are explanatory variables in regression always considered non-stochastic? If the explanatory variables are random or stochastic will the regression be still valid? What are the implications on the ...
learner's user avatar
  • 617
2 votes
1 answer
271 views

Meaning of & intuition behind predictors being fixed in linear regression [duplicate]

My question is a bit naive. I'm trying to get the exact & clear meaning of the phrase "predictor variables are fixed and not random in linear regression". According to my understanding, ...
SRISHTI GUREJA's user avatar
2 votes
0 answers
30 views

Regression: What is the difference between assuming the covariates are random or not random? [duplicate]

I often see regression expressed in two ways. The covariates are random: In this scenario, we have $(x_i,y_i) \sim G$ for some distribution $G$ and are i.i.d. for $i = 1, \cdots, n$. We then posit $...
michael_fortunato's user avatar
39 votes
6 answers
11k views

Under which assumptions a regression can be interpreted causally?

First, don't panic. Yes, there are many similar question on this site. But I believe none gives a conclusive answer to the question below. Please bear with me. Consider a data generation process $\...
luchonacho's user avatar
  • 2,758
21 votes
5 answers
4k views

Definition and delimitation of regression model

An embarrassingly simple question -- but it seems it has not been asked on Cross Validated before: What is the definition of a regression model? Also a support question, What is not a regression ...
Richard Hardy's user avatar
22 votes
2 answers
7k views

What is the difference between conditioning on regressors vs. treating them as fixed?

Sometimes we assume that regressors are fixed, i.e. they are non-stochastic. I think that means all our predictors, parameter estimates etc. are unconditional then, right? Might I even go so far that ...
Hirek's user avatar
  • 1,007
7 votes
2 answers
3k views

What does the assumption: "The independent variable is not random." in OLS mean?

What does the assumption: "The independent variable is not random." in OLS mean? How can you verify that hypothesis?
Victor's user avatar
  • 1,045
7 votes
3 answers
673 views

Which likelihood function is used in linear regression?

When trying to derive the maximum likelihood estimation for a linear regression, We start by a likelihood function. Does it matter if we use either of these 2 forms? $P(y|x,w)$ $P(y,x|w)$ All pages ...
floyd's user avatar
  • 1,382
3 votes
2 answers
2k views

Regression and the CEF

I recently read in this page (https://www.timlrx.com/2018/02/26/notes-on-regression-approximation-of-the-conditional-expectation-function/#fn1) that: "Regression offers a way of approximating ...
Rafael Hernández Salazar's user avatar
1 vote
2 answers
1k views

Regression's population parameters

Suppose I've specified a linear regression model: $$ Y = \beta_0 + \beta_1 X + \epsilon $$ where $\beta_0$, $\beta_1$ are the population parameters. My question is: why are these parameters ...
Juan Bromas's user avatar
5 votes
1 answer
2k views

Proof that Regression Sum of Squares and Residual Sum of Squares are independent random variables

Having consulted a number of sources, I still can't find a complete proof that Regression Sum of Squares ($SS_{regression}$) and ($SS_{residual}$) are independent random variables. I'll be doubly ...
ColorStatistics's user avatar
4 votes
2 answers
1k views

Does the OLS estimator in simple linear regression converge a.s.?

Consider the following model. Assume $(x_i, u_i)$ is sequence of independent identically distributed random vectors in $\mathbf{R}^{d+1}:$ $x_i$ are $\mathbf{R}^d$-value random vectors, which will ...
William M.'s user avatar
2 votes
0 answers
2k views

Do fixed design prediction/estimation error guarantees translate to random design for linear regression? When and How? [closed]

Suppose I have an independent vector $X$ and a dependent scalar random variable $Y$ and I wish to construct a regression model to predict $Y$ using $X$ given data $\{(x_i,y_i)\}_{i=1}^{n}$. For ...
ProAmateur's user avatar
2 votes
2 answers
298 views

In least square linear regression model, why does the test t-statistic of $\hat{\beta}$ follow a t distribution?

In the least square linear regression model, if the explanatory variables and the error term are independent, and the error term is normal, why does the t-statistic of $\hat{\beta}$ follow a t ...
denizen of the north's user avatar

15 30 50 per page