Questions tagged [bias]
The difference between the expected value of a parameter estimator & the true value of the parameter. Do NOT use this tag to refer to the [bias-term] / [bias-node] (ie the [intercept]).
1,084
questions
0
votes
0
answers
3
views
Can we get the conditional bias of the estimator at a generic $x$?
Consider a standard ERM problem based on quadratic loss where we solve
$$
\hat{f}_n\in \arg min_{f\in \mathcal{F}} R_{tr}(f)
$$
where $R_{tr}(f)=\frac{1}{n}\sum_{i=1}^n (Y_i-f(X_i))^2$, $\{(X_i, Y_i): ...
3
votes
3
answers
133
views
Do autocorrelated residuals cause OLS coefficients to be biased?
I see different answers everywhere. Intuitively, I would think if residuals are autocorrelated then there is some information that you are not incorporating into your model and is a sign of a biased ...
0
votes
0
answers
27
views
Derivation of bias of LASSO in the ortnormal case
In the following lecture slides by Breheny, P. (2016) titled "Adaptive lasso, MCP, and SCAD" from the High Dimensional Data Analysis course at the University of Iowa, slide 2 presents the ...
6
votes
5
answers
339
views
Name of this fallacy and how to reach conclusion
While handling some demographic data, I stuck in a position where (I did not disclose the actual data set and whom it is concerning, therefore I replace it with hypothetical data) I could not reach a ...
2
votes
1
answer
56
views
Check if method of moments estimator is unbiased for $X_1...X_n$ being a random sample from $\mathcal{U}_{[-\theta,\theta]}$
I am not sure how to do this. To find the method of moments estimator I did:
$$E[X] = \frac{-\theta + \theta}{2} = 0$$
use 2nd moment:
$$E[X^2] = \frac{(-\theta)^2 + -(\theta^2) + \theta^2}{3} = \frac{...
0
votes
0
answers
37
views
Systematic bias of neural network regression
I was trying to do graph-level regression task using graph convolutional networks, basically I concatenated 3 linear layers after several GCN modules, I used ReLU activation function before each ...
7
votes
1
answer
62
views
On unbiasedness of an optimal forecast
Diebold "Forecasting in Economics, Business, Finance and Beyond" (v. 1 August 2017) section 10.1 lists absolute standards for point forecasts, with the first one being unbiasedness: Optimal ...
3
votes
2
answers
84
views
Instrumental variable as a control variable
I understand that instrumental variable is used to address endogeneity bias since there could be correlation between the variable of interest and the error term.
Suppose now we want to see the ...
1
vote
0
answers
20
views
Multiplicative BIASES in Log-Log regression
When we try to estimate elasticities by regression, we usually estimate the following regression model:
$$ln(y) = \beta_0 + \beta_1 ln(x_1) + \dots + \epsilon$$
When we expect to have endogenous ...
2
votes
1
answer
90
views
How does non-collapsibility and the lack of an error term affect coefficients in regression
I have read from here that in nonlinear models such as the logit and Cox, because of a lack of an error term, coefficients may be biased (typically towards zero) when covariates are omitted; I see how ...
4
votes
2
answers
73
views
What does it mean that BLUP is unbiased, given a linear two-level model?
Suppose we have the following mixed effects model for observation $Y_{ij}$ of pupil $i$ in school $j$:
$Y_{ij}=b_0 + u_j + e_{ij}$
Here, $b_0$ is a fixed parameter for the "grand mean", $u_j$...
0
votes
1
answer
54
views
Bias vs consistency in instrumental variable estimation
So in Mostly Harmless Econometrics, page 154, they analyse the bias of instrumental variables:
They consider the case of one endogenous variable $x$, multiple instruments $Z$, and $\eta$ is the ...
0
votes
0
answers
25
views
Treating longitudinal data as a repeated cross section
Can you introduce bias by treating longitudinal data as a repeated cross section?
Suppose I have two data sources measuring the same variables. The first is a balanced panel dataset $\{y^{long}_{it},X^...
0
votes
0
answers
27
views
Why does the jackknife reduce bias? [duplicate]
Given a sample $x = (x_1, \ldots, x_n)$, define $x_{(-i)}$ as the sample values excluding sample $x_i$. That is,
$$
x_{(-i)} = (x_1, \ldots, x_{i-1}, x_{i+1}, \ldots x_n).
$$
Now given estimator $T(x)$...
0
votes
0
answers
50
views
Why the MSE of the fitted data is not equal to the sum of the bias and the variance in R?
I use simple linear regression and I want to find the decomposition of MSE, that is as a sum of the bias, the variance and the variance of the error terms. I have the following code:
...
1
vote
1
answer
25
views
Using Forecasted Data to Augment Predictions
We have a model that is predicting 5 year rent growth. We know that supply for the next two years is at a record high. We know that this record high supply is going to impact the rent growth ...
4
votes
1
answer
186
views
Resample from a sample to match a desired distribution
Suppose I have observations $x_1,\dots,x_n$, sampled iid from some distribution on $\mathbb{R}$, with pdf $p(x)$. Suppose I wish I had a sample from the distribution with pdf $q(x)$. Is there a way ...
1
vote
1
answer
40
views
For a biased estimator, how does one call the point for which the expected value of the estimator is equal to the observed sample estimate? [closed]
Let $\hat{\theta}$ be a biased estimator whose bias depends on the true value $\theta_0$, such that $E[\hat\theta|\theta_0]= f(\theta_0)\neq \theta_0$. Let $t_{sample}$ be a sample realization of $\...
1
vote
0
answers
35
views
Expectile loss to reduce dependent variables overestimation
Say I have a a bunch of covariates $X$, and a dependent variable $y$, where $y$ is collected from people. However, I know from psychology that people will tend to overestimate $y$ given $X$ in some ...
2
votes
0
answers
60
views
Is this a correct explanation of the asymptotic bias of maximum likelihood?
I want to be sure I understand, so please critique the following:
In regular parametric statistical models, the non-linear maximum likelihood estimator is biased. Given some data, $y_i$, parameters, $...
2
votes
1
answer
61
views
Data taken from survey where survey-takers self report a continous variable
I have a problem with some health data that I'm trying to analyze. The main issue originates from a census variable is derived from self reported times. The variable is sleep duration, which is ...
2
votes
2
answers
133
views
Why is the asymptotic bias of the maximum likelihood estimate $b(\theta) = \frac{b_1(\theta)}{n}+\frac{b_2(\theta)}{n^2}+...$?
Firth (1993) states in his introduction that for a $p$-dimensional parameter $\theta$ the asymptotic bias of the maximum likelihood estimate $\hat{\theta}$ may be written as:
$b(\theta) = \frac{b_1(\...
0
votes
0
answers
12
views
Statistical analysis to interpret beta effect size for two different elastic net model
I have two elastic net model and I want to compare their coefficient to say if they have any significant beta effect changes across these two models.
I thought of using Anova but realized since we don'...
2
votes
0
answers
34
views
Best way to address selection bias when outcome cannot be randomized
I have an (low incidence) binary outcome compared between 2 groups. The intervention for group 1 is coming from a specific type of center (academic) while group 2 from a different center. It is not ...
0
votes
0
answers
26
views
Test for Look-ahead bias in Time Series Forecasting
I have a general question regarding testing for look-ahead bias. Is there any technical test for look-ahead bias in training data? Especially in the context of time series forecasting e.g. predicting ...
0
votes
0
answers
8
views
Notable changes when modeling unevenly univariate spaced time series as an evenly spaced multivariate time series?
When attempting to model univariate data (although, this could easily be extended to the multivariate case) that is unevenly spaced over time, a natural approach to be able to apply common time series ...
1
vote
1
answer
92
views
Difference between consistent and unbiased estimator [duplicate]
I have a problem where I have to think of an example to explain a practical example of consistency and unbiased. The example I thought of is the sample mean.
Consistency is when the estimator (sample ...
1
vote
0
answers
49
views
Proof of attenuation bias in multiple linear regression model
Consider the case of measurement error with a single explanatory variable measured with error
\begin{equation}
y=\beta_0 + \beta_1 x_1 + \beta_2 x_2 + ... + \beta_k x^{\ast}_k + \nu
\label{...
1
vote
1
answer
50
views
Is it possible to use poststratification when some observations have missing values on the variables used as strata?
This is a theoretical question, so I don't have data to share.
Let's say I know the percentage of men and women in my population of interest, as well as the distribution of occupations and age ...
1
vote
0
answers
33
views
Providing biased data to analyst -- how could they use information about the bias? [closed]
I'm working on a project that provides an anonymized dataset to a service. Clients of this service will often use our data to make inferences to the population it's drawn from.
Our data is known to be ...
1
vote
0
answers
42
views
Regress y on residuals of x and z [closed]
I have the following set up:
$y_i = \beta_0 + \beta_1 x_i + \beta_2 z_i + e_i$, where $e_i$ is extracted from a Normal (0,1) distribution independently of $x$ and $z$. The true values are $\beta_0 = \...
0
votes
0
answers
33
views
Determine direction of bias with measurement error
We want to estimate the following population model:
$$y_i=\beta x_i+\epsilon_i$$
with $E[y_i]=E[x_i]=$ and $E[x_i\epsilon_i]=0$. We cannot observe $x_i$ directly, but we observe two variables $x_i^a$ ...
0
votes
0
answers
25
views
Bias in survival analysis: nice summary of all the different types and remedies
I am quite new to the field of survival analysis and am getting lost and confused with all the different types of bias that can occur, particularly in observational studies. For example, it appears ...
0
votes
0
answers
22
views
Is there any bias introduced by evaluating a model and decisions based on this model on the same data set?
As an example, let's say we have some financial time series such as closing prices of some stock and we would like to evaluate the ability of different models to forecast future closing prices as well ...
4
votes
2
answers
134
views
Variable selection with a theoretical DAG vs algorithmically discovered DAG
I'm analysing data from an electronic health record and determining what variables to include in a model to close back doors and omit bias.
I've read that it is important to have a subject specific ...
1
vote
0
answers
21
views
What type of bias is this?
I have a longitudinal cohort study, with individuals that diagnosed with Disease A at "start", and they all developed Complication B at "end" in my study period (my study period ...
0
votes
1
answer
57
views
How to compare two multivariate distribution (of distances) to zero in terms of mean and variance in R?
We have N 3D coordinates estimated with two methods and want to compare them with a reference set of N 3D coordinates which is the ground truth, so in notations:
...
2
votes
1
answer
232
views
mlogit + logitr packages fail to recover true estimates of mixed logit random coefficient model
I am running Monte-Carlo simulations on a simple DGP of a mixed logit random coefficient model to check if the mlogit and logitr ...
3
votes
1
answer
71
views
Tossing Until First Heads Outcome, and Repeating, as a Method for Estimating Probability of Heads
Consider the problem of estimating the heads probability $p$ of a coin
by tossing it until the first heads outcome is observed. Say we get $k_1$
tosses, then $U_1 = \frac{1}{k_1}$ is an estimate for $...
0
votes
1
answer
19
views
Censoring and then re-entering subjects
I am following a group of woman during the study period and performing the analysis using Cox model, comparing non-users against users of the investigated medicine. However, to remove the impact of ...
0
votes
1
answer
47
views
Analysis of the bias resulting from PCA [closed]
Suppose that we generate some dataset from $y = X \beta + \epsilon,$ where $\epsilon$ is some independent error, and the rows of $X$ come from some distribution (unspecified for now). Suppose you run ...
2
votes
0
answers
49
views
Bias and Variance of a Honest Random Forest
I am trying to read the paper Estimation and Inference of Heterogeneous Treatment
Effects using Random Forests. In the section 3.1(Theoretical Background), page 13 paragraph 2, The authors have ...
1
vote
1
answer
39
views
Are missing variables an important factor when considering instrumental variable analysis?
I'm currently reading some papers that deal with the effects of education on health (smoking and obesity). Mostly they use an IV approach (college availability).
However in several analysis, only a ...
2
votes
2
answers
127
views
Proof of the bias-variance decomposition in Bishop's book
I am trying to rewrite the demonstration given in Bishop's book: Pattern Recognition and
Machine Learning (2009)
I reproduce the figure (page 149) in which I am unclear about the step leading from (3....
0
votes
0
answers
8
views
How to improve sample representativeness for longitudinal data collected via an online platform?
I am working with a longitudinal dataset exploring cognitive ageing (e.g., memory performance over time). Participants complete the study annually. Inclusion criteria for this study are 1) UK resident,...
3
votes
1
answer
331
views
Question about Analogy to Statistics
If anyone could help me verify if my analogy is correct, thanks so much!
Here is an analogy:
A population is like a pot of soup.
We stir the pot of soup with the ladle because naturally the contents ...
1
vote
0
answers
16
views
Can I use Shapley values with metadata (i.e. information about observations that I didn't train my model on)?
I'm training a set of models (random forest/XGBoost) for an ordinal regression task. I'm (tentatively) planning to use Shapley values to infer feature performance.
I also have some metadata that my ...
2
votes
1
answer
88
views
How to determine whether a sample from a known population is significantly biased?
I have a large dataset (the population) and a large subset of it (the sample) containing the same, continuous variables. The sample represents more than 90% of the population but is not random -- we ...
2
votes
2
answers
300
views
Conceptually, what is the bias of the standard error of an estimator?
I'm reading Muthén and Muthén (2002) to learn how to use Monte Carlo simulation to estimate statistical power in regards to the coefficients of a model that is linear in its coefficients.
I understand ...
2
votes
1
answer
45
views
How normalizing data cause not problem in prediction?
In algorithms that perform better with data normalization or deep learning problems such as classification, how normalizing data does not bias our algorithm? I mean, in training or even testing, we ...