135 views

### Regression with random X [duplicate]

Suppose we have a standard regression model $$Y= X\beta + \epsilon$$ with $$\epsilon \sim \sigma^2$$ $$X \sim N(\mu,\gamma^2)$$ Are the estimated coefficients the same as if $X$ was fixed? Is ...
176 views

### Random Explanatory / Independent Variables [duplicate]

Are explanatory variables in regression always considered non-stochastic? If the explanatory variables are random or stochastic will the regression be still valid? What are the implications on the ...
27 views

2k views

### Definition and delimitation of regression model

An embarrassingly simple question -- but it seems it has not been asked on Cross Validated before: What is the definition of a regression model? Also a support question, What is not a regression ...
4k views

### What is the difference between conditioning on regressors vs. treating them as fixed?

Sometimes we assume that regressors are fixed, i.e. they are non-stochastic. I think that means all our predictors, parameter estimates etc. are unconditional then, right? Might I even go so far that ...
313 views

### Which likelihood function is used in linear regression?

When trying to derive the maximum likelihood estimation for a linear regression, We start by a likelihood function. Does it matter if we use either of these 2 forms? $P(y|x,w)$ $P(y,x|w)$ All pages ...
541 views

351 views

### Regression's population parameters

Suppose I've specified a linear regression model: $$Y = \beta_0 + \beta_1 X + \epsilon$$ where $\beta_0$, $\beta_1$ are the population parameters. My question is: why are these parameters ...
880 views

### When can we use fixed design regression results for the random design setting? [closed]

Suppose I have an independent vector $X$ and a dependent scalar random variable $Y$ and I wish to construct a regression model to predict $Y$ using $X$ given data $\{(x_i,y_i)\}_{i=1}^{n}$. For ...
223 views

### Why does regression model theory not use measure-theoretic sigma-field type notation but counting process models do?

I have been studying counting process theory for time to recurrent event processes and am interested in the explicit use of the conditioning set in the model notation; E[dN(t)|\mathcal{F}_{t^{-}}]=\...
129 views

### Foundations behind Linear Regression / Statistical Modelling

I've always struggled with the foundations behind the concept of modelling (and specifically regression) - what is random, what is not, what we are modelling. I think I have a grasp of it - but I'd ...
340 views

### non stochastic regressors and causation

Randomized controlled experiment is base case for causality (also) in regression. However currently I’m analyzing the role of causality in linear regression as shown in many econometrics textbook. ...
184 views

### Proof that Regression Sum of Squares and Residual Sum of Squares are independent random variables

Having consulted a number of sources, I still can't find a complete proof that Regression Sum of Squares ($SS_{regression}$) and ($SS_{residual}$) are independent random variables. I'll be doubly ...
129 views

### Does the OLS estimator in simple linear regression converge a.s.?

Consider the following model. Assume $(x_i, u_i)$ is sequence of independent identically distributed random vectors in $\mathbf{R}^{d+1}:$ $x_i$ are $\mathbf{R}^d$-value random vectors, which will ...
79 views

### Confused with the fundamental assumptions of Frequentist and Bayesian Linear Regression

In Frequentist Linear Regression, I have seen 2 approaches which lead to basically similar models. We have $W,y,X,\epsilon$ related as $y=W^TX+\epsilon$, where $y$ is the dependent random variable, ...
76 views

### Clarification on the assumptions $E[u|x]=0$ and the $x_i$ being fixed in repeated samples in Wooldridge Introductory Econometrics

The author is writing on the assumption $E[u|x]=0$. The part of the text which is not clear to me is this (the red lines emphasize where the critical portions are located) : In the first piece I don'...
131 views

### Non-stochastic vs Stochastic regressors and sampling distributions and causation?

I was wondering if I understand these correctly. Would an example of a stochastic regressor be weather? so when thinking about the sampling distribtuion and causality, I would think of repeated ...
104 views

### In least square linear regression model, why does the test t-statistic of $\hat{\beta}$ follow a t distribution?

In the least square linear regression model, if the explanatory variables and the error term are independent, and the error term is normal, why does the t-statistic of $\hat{\beta}$ follow a t ...
45 views

### Deriving the posterior distribution over the model parameters: are the model parameters and data independent?

We are told (in Section 9.2.3, Deisenroth et al.: Mathematics for Machine Learning) that we can compute the posterior over a model's parameters $\boldsymbol\theta$ (here in the context of linear ...
73 views

### Why is the likelihood function for linear regression based on $𝑝(𝑦|𝑥)$ instead of $𝑝(𝑥,𝑦)$

Likelihood functions typically based on the joint probabilities of the variables involved. In linear regression we have variables 𝑥 and 𝑦, but derivations for MLE under a zero meaned gaussian ...
### Do we assume that the regressors are uncorrelated with the unobserved error $\epsilon$ for least squares?
I recall seeing sources in the past state that the Gauss-Markov assumptions assume that the regressors are uncorrelated with $\epsilon$ in order to make $E[\hat{\beta}] = \beta$. But is this ...
I see that in Bayesian neural networks likelihood function is defined in two ways: $p(W|D) = Z^{-1} p(D|W)p(W)$ or $p(W|y,x)=Z^{-1}p(y|x,W)p(W)$ Are there a slight difference in interpreting $p(D|W)$ ...