505 views

### Regression with random X [duplicate]

Suppose we have a standard regression model $$Y= X\beta + \epsilon$$ with $$\epsilon \sim \sigma^2$$ $$X \sim N(\mu,\gamma^2)$$ Are the estimated coefficients the same as if $X$ was fixed? Is ...
1 vote
369 views

### Random Explanatory / Independent Variables [duplicate]

Are explanatory variables in regression always considered non-stochastic? If the explanatory variables are random or stochastic will the regression be still valid? What are the implications on the ...
184 views

### Meaning of & intuition behind predictors being fixed in linear regression [duplicate]

My question is a bit naive. I'm trying to get the exact & clear meaning of the phrase "predictor variables are fixed and not random in linear regression". According to my understanding, ...
29 views

3k views

### Definition and delimitation of regression model

An embarrassingly simple question -- but it seems it has not been asked on Cross Validated before: What is the definition of a regression model? Also a support question, What is not a regression ...
6k views

### What is the difference between conditioning on regressors vs. treating them as fixed?

Sometimes we assume that regressors are fixed, i.e. they are non-stochastic. I think that means all our predictors, parameter estimates etc. are unconditional then, right? Might I even go so far that ...
3k views

### What does the assumption: "The independent variable is not random." in OLS mean?

What does the assumption: "The independent variable is not random." in OLS mean? How can you verify that hypothesis?
568 views

### Which likelihood function is used in linear regression?

When trying to derive the maximum likelihood estimation for a linear regression, We start by a likelihood function. Does it matter if we use either of these 2 forms? $P(y|x,w)$ $P(y,x|w)$ All pages ...
2k views

1 vote
980 views

### Regression's population parameters

Suppose I've specified a linear regression model: $$Y = \beta_0 + \beta_1 X + \epsilon$$ where $\beta_0$, $\beta_1$ are the population parameters. My question is: why are these parameters ...
1k views

### Proof that Regression Sum of Squares and Residual Sum of Squares are independent random variables

Having consulted a number of sources, I still can't find a complete proof that Regression Sum of Squares ($SS_{regression}$) and ($SS_{residual}$) are independent random variables. I'll be doubly ...
828 views

### Does the OLS estimator in simple linear regression converge a.s.?

Consider the following model. Assume $(x_i, u_i)$ is sequence of independent identically distributed random vectors in $\mathbf{R}^{d+1}:$ $x_i$ are $\mathbf{R}^d$-value random vectors, which will ...
2k views

### Do fixed design prediction/estimation error guarantees translate to random design for linear regression? When and How? [closed]

Suppose I have an independent vector $X$ and a dependent scalar random variable $Y$ and I wish to construct a regression model to predict $Y$ using $X$ given data $\{(x_i,y_i)\}_{i=1}^{n}$. For ...
397 views

### Why does regression model theory not use measure-theoretic sigma-field type notation but counting process models do?

I have been studying counting process theory for time to recurrent event processes and am interested in the explicit use of the conditioning set in the model notation; E[dN(t)|\mathcal{F}_{t^{-}}]=\...
174 views

### Foundations behind Linear Regression / Statistical Modelling

I've always struggled with the foundations behind the concept of modelling (and specifically regression) - what is random, what is not, what we are modelling. I think I have a grasp of it - but I'd ...
249 views

### Clarification on the assumptions $E[u|x]=0$ and the $x_i$ being fixed in repeated samples in Wooldridge Introductory Econometrics

The author is writing on the assumption $E[u|x]=0$. The part of the text which is not clear to me is this (the red lines emphasize where the critical portions are located) : In the first piece I don'...
408 views

### non stochastic regressors and causation

Randomized controlled experiment is base case for causality (also) in regression. However currently I’m analyzing the role of causality in linear regression as shown in many econometrics textbook. ...
405 views

### Why is each observation in a sample considered a random variable in linear regression?

I have the following excerpt in my statistics textbook: I am confused by the sentence: "Another way statisticians treat this model is that, assume $X_1...X_n$ are random variables, we make ...
163 views

### Confused with the fundamental assumptions of Frequentist and Bayesian Linear Regression

In Frequentist Linear Regression, I have seen 2 approaches which lead to basically similar models. We have $W,y,X,\epsilon$ related as $y=W^TX+\epsilon$, where $y$ is the dependent random variable, ...
117 views

### Deriving the posterior distribution over the model parameters: are the model parameters and data independent?

We are told (in Section 9.2.3, Deisenroth et al.: Mathematics for Machine Learning) that we can compute the posterior over a model's parameters $\boldsymbol\theta$ (here in the context of linear ...
1 vote
225 views

### In least square linear regression model, why does the test t-statistic of $\hat{\beta}$ follow a t distribution?

In the least square linear regression model, if the explanatory variables and the error term are independent, and the error term is normal, why does the t-statistic of $\hat{\beta}$ follow a t ...
208 views

### Non-stochastic vs Stochastic regressors and sampling distributions and causation?

I was wondering if I understand these correctly. Would an example of a stochastic regressor be weather? so when thinking about the sampling distribtuion and causality, I would think of repeated ...
1 vote
119 views

### Why is the likelihood function for linear regression based on $𝑝(𝑦|𝑥)$ instead of $𝑝(𝑥,𝑦)$

Likelihood functions typically based on the joint probabilities of the variables involved. In linear regression we have variables 𝑥 and 𝑦, but derivations for MLE under a zero meaned gaussian ...
64 views

### Notation of the Likelihood Term in Bayesian Neural Networks

I see that in Bayesian neural networks likelihood function is defined in two ways: $p(W|D) = Z^{-1} p(D|W)p(W)$ or $p(W|y,x)=Z^{-1}p(y|x,W)p(W)$ Are there a slight difference in interpreting $p(D|W)$ ...
### Do we assume that the regressors are uncorrelated with the unobserved error $\epsilon$ for least squares?
I recall seeing sources in the past state that the Gauss-Markov assumptions assume that the regressors are uncorrelated with $\epsilon$ in order to make $E[\hat{\beta}] = \beta$. But is this ...
I am reading "Probabilistic Machine Learning" by K. Murphy. In it, he defines the likelihood of a dataset as However, this dataset $D$ is defined as: So if all $x_n, y_n$ are random ...