Questions tagged [bias]
The difference between the expected value of a parameter estimator & the true value of the parameter. Do NOT use this tag to refer to the [bias-term] / [bias-node] (ie the [intercept]).
1,092
questions
0
votes
0
answers
14
views
Standard practice to show Biased CRBs
I have a problem with four-parameter estimation. I have derived the variances for the estimated parameters using Monte Carlo simulations (numerical ones) and theoretical ones using the inverse of the ...
- 275
0
votes
1
answer
9
views
Does the intuitive sense of overfitting in this mechanism design context exemplify bias-variance tradeoff?
Suppose the (we can say unanimous) preference of each individual in a society is to select roads for travel by placing 95% weight on the objective of minimizing travel time, and the remaining 5% ...
- 9,327
1
vote
0
answers
28
views
Degrees of freedom for biased sample autocorrelation function
I want to find the expression for the a biased estimate of the autocorrelation function for a time series $X$, and am doing this from the biased estimated autocovariance function for lag $k$, divided ...
- 752
0
votes
0
answers
10
views
Worst case for propensity adjustment
I am studying various methods for CATE estimation that fall under causal ML (x/t/s/r-learners, and causal forests). I am wondering if there is a scenario which is known to be hard in terms of ...
- 1,355
0
votes
1
answer
268
views
How to quantify a bias with a score value (e.g. RMSE)
In machine learning, when are using some validation methods (e.g. CV or Bootstrap methods) to evaluate the performance of the machine learning algorithms. Apart from the prediction accuracy, other ...
CommunityBot
- 1
0
votes
1
answer
1k
views
Incidental parameters problem, bias direction, and robust standard errors
Incidental parameters problem results in away-from-zero biased estimates per Greene (2004). Okay. But can this bias result in a directional change as well, e.g., true value is +2.13 but estimate is -1....
CommunityBot
- 1
0
votes
0
answers
15
views
conditional-on-positives bias
I am reading the Bad COP section on https://matheusfacure.github.io/python-causality-handbook/07-Beyond-Confounders.html#bad-cop. I am confused if
$$
E[Y|T = 1] - E[Y|T = 0] = \\
E[Y|Y > 0, T = 1]...
- 78.1k
0
votes
0
answers
15
views
How to calculate bias having three groups?
Three groups of people each tried one of the three different applications and answered a questionnaire on a Likert scale from 0 to 4. Their age and experience in video games were also asked (on a ...
1
vote
1
answer
1k
views
How to calculate Bias and Variance to get the MSE value step by step?
I want to compute my MSE value for a forecast step by step for test set.
For me the Bias is:
Bias = mean(predicted values - actual values)
Variance = mean((predicted values- actual values)^2)
...
CommunityBot
- 1
1
vote
1
answer
27
views
Is Assessment Bias a type of Observer Bias?
Based on the definitions of assessment bias and observer bias I have found bellow, seems like assessment bias is a type of observer bias?
Assessment bias:
If the observer knows the treatment being ...
- 120k
0
votes
0
answers
9
views
comparing estimators for variance, cofusion about biases and mean square errors
I wanted to compare biases and mean square errors for three estimators of variance. I took one unbiased (first) estimator and two biased.
...
2
votes
1
answer
31
views
Can we get the conditional bias of the estimator at a generic $x$?
Consider a standard ERM problem based on quadratic loss where we solve
$$
\hat{f}_n\in \operatorname*{arg min}_{f\in \mathcal{F}} R_\text{tr}(f)
$$
where $R_\text{tr}(f)=\frac{1}{n}\sum_{i=1}^n (Y_i-f(...
- 125k
5
votes
3
answers
404
views
Do autocorrelated residuals cause OLS coefficients to be biased?
I see different answers everywhere. Intuitively, I would think if residuals are autocorrelated then there is some information that you are not incorporating into your model and is a sign of a biased ...
- 125k
1
vote
0
answers
19
views
Regression Discontinuity Design, staggered treatment allocation
I'm unsure if this complex allocation rule is appropriate for RDD. I will have data for a staggered rollout treatment where there will be about 10 rounds of selection over two years for services (...
- 362
0
votes
0
answers
31
views
Derivation of bias of LASSO in the ortnormal case
In the following lecture slides by Breheny, P. (2016) titled "Adaptive lasso, MCP, and SCAD" from the High Dimensional Data Analysis course at the University of Iowa, slide 2 presents the ...
- 8,797
4
votes
2
answers
607
views
Including Collider Variables in Prediction
When the goal is to estimate a causal association between X and Y in the regression framework, one should not condition on (include as covariates) collider variables (common causes of both X and Y) ...
6
votes
5
answers
358
views
Name of this fallacy and how to reach conclusion
While handling some demographic data, I stuck in a position where (I did not disclose the actual data set and whom it is concerning, therefore I replace it with hypothetical data) I could not reach a ...
- 78.5k
3
votes
1
answer
508
views
Omitted variable bias formula for 3 variable regression
Suppose our true model is:
$Y=\alpha +\beta_{1} X_{1}+\beta_{2} X_{2}+\beta_{3} X_{3}+u$
but instead, we omit $X_{3}$ and estimate the following by OLS:
$Y=\alpha +\beta_{1} X_{1}+\beta_{2} X_{2}+v$...
CommunityBot
- 1
4
votes
2
answers
4k
views
What is the relationship between estimation error, approximation error, bias, variance in machine learning?
I'm a beginner in machine learning. I was reading http://ciml.info/ 5.9 Bias/Variance Trade-off
According to this book:
The trade-off between estimation error and approximation error is often called ...
- 46
3
votes
2
answers
90
views
Instrumental variable as a control variable
I understand that instrumental variable is used to address endogeneity bias since there could be correlation between the variable of interest and the error term.
Suppose now we want to see the ...
- 563
2
votes
1
answer
56
views
Check if method of moments estimator is unbiased for $X_1...X_n$ being a random sample from $\mathcal{U}_{[-\theta,\theta]}$
I am not sure how to do this. To find the method of moments estimator I did:
$$E[X] = \frac{-\theta + \theta}{2} = 0$$
use 2nd moment:
$$E[X^2] = \frac{(-\theta)^2 + -(\theta^2) + \theta^2}{3} = \frac{...
- 706
0
votes
0
answers
39
views
Systematic bias of neural network regression
I was trying to do graph-level regression task using graph convolutional networks, basically I concatenated 3 linear layers after several GCN modules, I used ReLU activation function before each ...
- 155
7
votes
1
answer
65
views
On unbiasedness of an optimal forecast
Diebold "Forecasting in Economics, Business, Finance and Beyond" (v. 1 August 2017) section 10.1 lists absolute standards for point forecasts, with the first one being unbiasedness: Optimal ...
- 67.6k
3
votes
0
answers
169
views
Linear model with biased estimator
Consider a linear regression model. Suppose that the estimator $\hat{\beta}$ for the vector of the parameters of the model $\beta$ is, for some reasons, biased. As a consequence:
$$E\left[\hat{\beta}\...
- 8,292
1
vote
1
answer
310
views
How error derivative becomes zero in gradient descent
Previous questions this & this does not answer my question
Code
...
CommunityBot
- 1
1
vote
1
answer
100
views
Bias of an estimator depends on whether you take expectation of the estimator or its inverse
(Please read until the end)
Consider two ways of writing the exponential distribution:
(A) $\frac{1}{\beta} e^{-\frac{x}{\beta}}$ and
(B) $\theta e^{-x\theta}$
If I estimate $\beta$ or $\theta$ ...
- 67.6k
2
votes
1
answer
1k
views
Estimating a VAR model via OLS
I am looking at Vilasuso (2001), who says that when using least-squares to estimate causality in mean, there is significant size distortion if the conditional variances are correlated.
My question ...
CommunityBot
- 1
2
votes
1
answer
567
views
IV changes the sign of exogenous variable
After implementing an IV probit model, the signs of many exogenous covariates' coefficients have been flipped, compared to those in the baseline probit model. These signs are now at odds with the past ...
CommunityBot
- 1
1
vote
0
answers
21
views
Multiplicative BIASES in Log-Log regression
When we try to estimate elasticities by regression, we usually estimate the following regression model:
$$ln(y) = \beta_0 + \beta_1 ln(x_1) + \dots + \epsilon$$
When we expect to have endogenous ...
- 165
3
votes
2
answers
306
views
The role of bias terms in binary recommender systems
I realize that a recommender system applied to, for example, the Movielens dataset needs to account for bias. That is, one needs to adjust for the varying popularity of movies, and that users have ...
CommunityBot
- 1
57
votes
5
answers
21k
views
When is a biased estimator preferable to unbiased one?
It's obvious many times why one prefers an unbiased estimator. But, are there any circumstances under which we might actually prefer a biased estimator over an unbiased one?
- 1
1
vote
1
answer
759
views
Strict exogeneity is violated: consequences
What are the consequences for the OLS estimator when weak exogeneity holds but strict exogeneity does not? Thank you.
CommunityBot
- 1
1
vote
3
answers
2k
views
Understanding bias and variance for different models over same dataset
Consider we have 1-D data generated by a polynomial of degree 5. Which will of thhe following give higher / lower bias and higher / lower variance?
Regression with linear basis functions
Regression ...
CommunityBot
- 1
2
votes
1
answer
545
views
Classical logistic regression VS Firth logistic regression: comparison in power
I understand that in case of separated data, logistic regression via ordinary MLE has an upward bias in the p values, which implies that any penalized MLE designed to reduce this bias will have more ...
CommunityBot
- 1
3
votes
1
answer
99
views
How does non-collapsibility and the lack of an error term affect coefficients in regression
I have read from here that in nonlinear models such as the logit and Cox, because of a lack of an error term, coefficients may be biased (typically towards zero) when covariates are omitted; I see how ...
4
votes
2
answers
78
views
What does it mean that BLUP is unbiased, given a linear two-level model?
Suppose we have the following mixed effects model for observation $Y_{ij}$ of pupil $i$ in school $j$:
$Y_{ij}=b_0 + u_j + e_{ij}$
Here, $b_0$ is a fixed parameter for the "grand mean", $u_j$...
- 1,144
10
votes
3
answers
6k
views
Batch normalization and the need for bias in neural networks
I've read that batch normalization eliminates the need for a bias vector in neural networks, since it introduces a shift parameter that functions similarly as a bias. As far as I'm aware though, a ...
- 11.8k
4
votes
1
answer
1k
views
Relation between bias and R-square
I am trying to understand relation between bias and R-squared value in linear regression.
High bias means that the model is underfit. By this I am assuming that the R-square d will be less.
So my ...
CommunityBot
- 1
0
votes
1
answer
55
views
Bias vs consistency in instrumental variable estimation
So in Mostly Harmless Econometrics, page 154, they analyse the bias of instrumental variables:
They consider the case of one endogenous variable $x$, multiple instruments $Z$, and $\eta$ is the ...
- 59.1k
0
votes
0
answers
25
views
Treating longitudinal data as a repeated cross section
Can you introduce bias by treating longitudinal data as a repeated cross section?
Suppose I have two data sources measuring the same variables. The first is a balanced panel dataset $\{y^{long}_{it},X^...
- 78.1k
2
votes
1
answer
383
views
Geometric mean appropriateness with bimodally distributed data
I am trying to find out whether the performance of the geometric mean of a distribution as a measure of its central tendency would be impaired by the distribution being multimodal.
For example, ...
CommunityBot
- 1
4
votes
1
answer
2k
views
How to calculate MSE in a quantile regression simulation study
I am working on a simulation study on quantile regression. So what I did is to simulate data based on a given model, which is different from the true underlying model of the data, in other words, a ...
CommunityBot
- 1
3
votes
1
answer
3k
views
Prove that omitted variable bias may lead to endogeneity
I am looking for a proof that omitted variable bias (OVB) in OLS regression may lead to endogeneity. I have found many examples here and out there on how to prove that a given parameter $b_{j}$ (where ...
- 8,292
1
vote
1
answer
554
views
3SLS with fixed effects and its interpretation issue
I am running empirical models based on 3SLS due to the potential endogeneity issue caused by simultaneity. So, I opted for 3SLS as my main analysis tool, and I am using reg3 command to analyze it. My ...
CommunityBot
- 1
1
vote
1
answer
1k
views
Correcting for auto-correlation when using a lagged DV in the regression
I am conducting a regression where in I have data at the quarterly level for 19 companies (I have data ranging from 2007-2019 so about 30-50 quarters for each company). My regression model in STATA ...
CommunityBot
- 1
3
votes
1
answer
139
views
Can an omitted random variable cause "omitted variable bias"?
Suppose we have a linear regression:
Y = mx + b
where X is the independent variable of interest, in this case "scoops of ice cream per order" at an ice cream shop, b is the error term, and Y is the ...
- 120k
1
vote
1
answer
313
views
Network learns bias during the first iterations if parameter initialization is not good
Andrej Karpathy in his blog post "A Recipe for Training Neural Networks" states that initialization is important for convergence. I get that but when he says:
init well. Initialize the final layer ...
CommunityBot
- 1
0
votes
0
answers
28
views
Why does the jackknife reduce bias? [duplicate]
Given a sample $x = (x_1, \ldots, x_n)$, define $x_{(-i)}$ as the sample values excluding sample $x_i$. That is,
$$
x_{(-i)} = (x_1, \ldots, x_{i-1}, x_{i+1}, \ldots x_n).
$$
Now given estimator $T(x)$...
- 121
0
votes
0
answers
50
views
Why the MSE of the fitted data is not equal to the sum of the bias and the variance in R?
I use simple linear regression and I want to find the decomposition of MSE, that is as a sum of the bias, the variance and the variance of the error terms. I have the following code:
...
1
vote
1
answer
25
views
Using Forecasted Data to Augment Predictions
We have a model that is predicting 5 year rent growth. We know that supply for the next two years is at a record high. We know that this record high supply is going to impact the rent growth ...
- 1,680