Bias, in a statistical framework, means that an estimate of a parameter has an expected value that is not equal to the actual parameter value. There is often a tradeoff between bias and variance - low variance estimators may be more biased than high variance ones.

learn more… | top users | synonyms

1
vote
3answers
52 views

What is statistical Bias?

I am asking this question in context to section 4.1 in this paper: security control methods for statistical Database (http://www.utdallas.edu/~muratk/courses/privacy08f_files/stat_database_sec.pdf) ...
1
vote
0answers
15 views

Unbiased Estimator of Days Until Completion?

I'm trying to get an estimate of average number of days until some event occurs (the event is guaranteed to eventually occur). I have some sample where this event has already occured for most ...
1
vote
0answers
13 views

Selection bias correction and a multinomial logit

I have a data set for a number of people making 2 decisions - where to live; and how many hours to work. For every observation with a non-zero amount of work, there is an observed wage. I've assigned ...
0
votes
0answers
10 views

Exposure classification bias, post-modeling

I am building a high-dimensional Bayesian spike-and-slab model to study the association between several organic compounds and a continuous outcome. The goal is to select the most influential compounds ...
0
votes
0answers
12 views

Down-sampling with building models (specifically random forests)

I was wondering if anyone had ever used down-sampling to build random forests with data that has unbalanced classes. Basically down-sampling samples (with replacement) x*min from the population where ...
1
vote
1answer
37 views

Calculating Bias from bivariate data

My data consists of one column of real temperature, and one column of calculated temperatures. I want a single number which quantifies the 'bias' in the real temperatures. Basically, if most of my ...
2
votes
0answers
35 views

Where did the words bias and variance come from? [closed]

I understand that a bias model is more relaxed while a model with a lot of variance is more flexible. but, where did these terms come from, and, why 'bias' and why 'variance'?
1
vote
0answers
42 views

Does lots of bias==underfitting, while lots of variance==overfitting?

From what I understand, there is a relationship between bias and underfitting; as well as variance and overfitting. Is a 'biased model' another word for an 'underfitted model'? Likewise, is a ...
3
votes
0answers
42 views

Bias Variance tradeoff from a Bayesian perspective

I know the general question about bias variance has been asked before. I understand the frequentist approach and the concept of model selection and the impact of bias and variance on "accuracy" of a ...
3
votes
1answer
158 views

Bias of the maximum likelihood estimator of an exponential distribution

The maximum likelihood estimator of an exponential distribution $f(x, \lambda) = \lambda e^{-\lambda x}$ is $\lambda_{MLE} = \frac {n} {\sum x_i}$; I know how to derive that by find the derivative of ...
2
votes
2answers
70 views

How to check if a random sample is biased

A question on how to prove that differences in mean value are statictically significant, and not just random noise. I have a set of two observations, one of which I'm going to deliberately bias like ...
2
votes
2answers
43 views

explanatory variables may bias predictions

I' m asking this question out of sheer curiosity, my teacher was not able to explain it. If I'm using logistic regression with categorical variables they are coded like {1,2,3}. I guess it wouldn't ...
3
votes
1answer
100 views

Differential baseline bias vs Heterogenous treatment effect

Where it says 'Differential baseline bias only', I see both a differential baseline bias and a differential treatment effect bias. From my understanding, to have no differential treatment effect bias, ...
3
votes
1answer
75 views

Is it possible for a regression to have the right functional form but still suffer from omitted variable bias?

Suppose that $$ Y = b + aX + e$$ where you know that $E[Y|X] = b + aX$. Is it true that the model cannot suffer from omitted variable bias? If this is true, then it follows that omitted variable ...
1
vote
0answers
50 views

Selection correction variable (non-randomly selected sample / bias) and Cox regression

I found an approach to calculate a selection correction variable for a Cox regression (Lee 1983. Generalized econometric models with selectivity. Econometrica 51: 507–512.): $$ λ = \phi \frac{ ...
0
votes
1answer
52 views

AR process with a constant

I am having trouble understanding the estimation of an AR process. In some textbooks, the AR(1) process is defined as follows: $y_{t}=\theta y_{t-1}+ϵ_t$ (which does not contain a constant). So the ...
1
vote
0answers
51 views

Bias Variance Decomposition for Mean Absolute Error

The mean squared error of an estimator $\hat{\theta}$ with respect to an unknown parameter $\theta$ is defined as $$ MSE(\hat{\theta})=E[(\hat{\theta}-\theta)^{2}]. $$ It is well known that there is ...
1
vote
1answer
59 views

Model fit is High but Ramsey RESET Test suggests omitted variables. What to do?

I'm trying to figure out what next steps to take. I created a model and ran OLS on a very large sample of data (over 400000 observations) and got an R-squared value of 0.80. So the model fit seems ...
0
votes
1answer
47 views

bias and variance of correlation estimator

I calculated the bias and variance of sample mean $\hat{\mu}$ and sample variance $\hat{\sigma}^2$ but I could not calculate the bias and variance for sample correlation $\hat{\rho}$. How can I do the ...
0
votes
1answer
50 views

Bias of more than one endogenous variables

I may have a model with omitted variables that are correlated with my predictor variables. If I have in my model, let's say, two endogeneous variables X1, X2, but I am interested in obtaining an ...
0
votes
0answers
42 views

diagnosing bias, variance from learning curve

I was doing machine learning course at coursera and there was a lecture about diagnosing high bias, or high variance from learning curves. If anyone interested here is the lecture - ...
3
votes
0answers
43 views

Comparing two distributions with different biased sampling criteria

Suppose you are running an experiment with two conditions, A and B. At the beginning of the experiment, both populations are the ...
1
vote
0answers
19 views

Drawing a conclusion from data without a correlation to other supporting data

I am looking for the word/phrase that is used to say that a (potential) erroneous conclusion has been drawn from data without having a correlation to other supporting data points. For example, if ...
2
votes
3answers
90 views

Can something show bias but not be significant?

I was reading a research paper and read this statement: ...
1
vote
1answer
55 views

Simulating a bimodal biased IV estimator

How can I simulate a bimodal biased IV estimator? The common unimodal heavy-tailed biased estimator would be something like this: ...
4
votes
0answers
32 views

R packages that work with biased samples

I'm working with a biased sample of web users. I'm only able to track responses of users who have navigated my site in a certain way, and I'd like to run an analysis to determine how certain factors ...
0
votes
1answer
62 views

Does BIAS equal to MEAN ERROR

Bias is defined as an average of all errors (without abs) and this is, IMO, what I want. However, I have been asked to give MEAN ERROR. Is this the same than bias and is it wrong to call bias as mean ...
1
vote
0answers
38 views

Forecasting and auto-correlation [duplicate]

I'm reading this chapter forecasting principles and practise from a forecasting book. The author has explained a linear regression model. Now this linear regression model will definitely have some ...
5
votes
1answer
86 views

statistical handling of lab values below limit of quantitation (BLQ)

There were several samples BLQ because of the lower limit of quantitation (LLQ) of the method, e.g. 5 ng/ml or less. Using the statistical program PRISM6 I marked these values together with the ...
1
vote
0answers
51 views

Comparison between normal glm and glm.nb regression with quadratic term?

Let's say I have a function to simulate data for negative binomial regression: ...
5
votes
1answer
115 views

How do instrumental variables address selection bias?

I'm wondering how an instrumental variable addresses selection bias in regression. Here's the example I'm chewing on: In Mostly Harmless Econometrics, the authors discuss and IV regression relating ...
0
votes
0answers
16 views

Calculating Bias from Cox PH model

I am using COX PH model to fit a lifetime data set and estimating parameters. By the by I am also simulating data and trying to find out estimates of parameters. Now I want to calculate bias and ...
0
votes
0answers
25 views

handling multicollinearity by backwards regression and omitted variable bias

Suppose I try to estimate a production function as follows: $logY=b_0+b1*logX_1+b_2*logX_2+b_{11}*(logX_1)^2+b_{22}*(logX_2)^2+b_{12}*(logX_1)*(logX_2)+u$, where $Y$ is the output, $X_1$, $X_2$ are ...
1
vote
0answers
25 views

Estimating the probability of causation based on finding a correlation, including experimental details

Say there is a hypothesis that A causes B (A -> B), and some likelihood that the hypothesis is correct (AB1%). Now, an experiment is run that claims to find a correlation between A and B. What I ...
1
vote
1answer
147 views

Why use d-prime instead of percent correct?

In signal detection theory, people often use $d'$ to assess performance. Apart from the fact that $d'$ is in $z$ units (units of measurement transformed to standard deviation units, i.e., $z$ scores), ...
0
votes
0answers
31 views

Finding parameter bias under omitted variable, with variance covariance notation

Dear CrossValidated community, Can anyone help me to prove the bias in a given parameter of a regression when there is omitted variable? I know to do it using matrices and matrix algebra. For ...
1
vote
0answers
78 views

Proof that omitted variable bias may lead to endogeneity

I am looking for a proof that omitted variable bias (OVB) in OLS regression may lead to endogeneity. I have found many examples here and out there on how to prove that a given parameter $b_{j}$ (where ...
1
vote
1answer
84 views

Is the bootstrap estimate of the mean biased when a single extreme value is in the sample?

My sample includes $n$ random observations, while $n-1$ of these observations are in the range (0-1) there is also one observation that gets very high value. For example, a sample of prices where ...
0
votes
0answers
48 views

Optimism bias - estimates of prediction error

The book Elements of Statistical Learning (available in PDF online) discusses the optimisim bias (7.21, page 229). It states that the optimism bias is the difference between the training error and the ...
1
vote
0answers
15 views

RTM and study design issue

I was asked to estimate "regression to the mean" (RTM) error for the purpose of measuring program efficacy for reducing spending among the highest spending individuals. Given the constraints of the ...
16
votes
3answers
460 views

Intuitive reasoning behind biased ML estimators

I have a confusion on biased Maximum Likelihood estimators. The mathematics of the whole concept is pretty clear to me but I cannot figure out the intuitive reasoning behind it. Given a certain ...
0
votes
0answers
24 views

Selection Bias with Groups

I have conducted a pre-post test study with a set of 20 participants. There were two independent variables and therefore 4 groups of 5 participants each. The experiment took place in a classroom and ...
0
votes
1answer
42 views

Longitudinal panel dataset: Consequences of missing values

I am analyzing a longitudinal panel dataset using OLS. The data spans around 40 years, but for some variables data was unavailable for certain categories. In most cases, the data for given category ...
2
votes
2answers
318 views

Bias and Variance - Errors in R example

I am trying to better understand the bias and variance trade-off, and tried to create a R example. It attempts to calculate the bias and variance of smoothing splines with different parameters. ...
2
votes
0answers
46 views

Bias and variance estimation with boostrap

The Wikipedia article about Jacknife estimation of the bias and variance of an estimator $\theta$ includes the following formulas: Variance of $\theta$: $ \operatorname {Var}(\theta )=\sigma ...
1
vote
1answer
58 views

Reason for not shrink the bias term

For linear model, $y=\beta_0+x*\beta+\varepsilon$, the shrinkage term is always like $P(\beta) $. What's the reason we do not shrink the bias term $\beta_0$? Comparatively, should we shrink the bias ...
1
vote
2answers
79 views

How to correct sample size bias in logistic regression?

I'm working on a logistic regression with N = 42, which is too small following Hosmer's and Lemeshow's recommendations of at least N = 100. How will this affect my logistic regression and is there a ...
0
votes
0answers
34 views

Adjusting regression for correlated errors-in-variables

I have a set of data points $\{x_i\}$. These data points are grouped so that (say) $i\in\{1,2,3\}$ is group $A$, $i\in\{4,5,6,7\}$ is group $B$, etc. I would like to test the null hypothesis of no ...
1
vote
1answer
74 views

Weird bootstrap bias for Predictor Importance (MeanDecreaseAccuracy) in Random Forests

Below, using R, I: 1. Create a data set with a bunch of factors. All of them are predictors and 'y' is the dependent variable. 2. I run a classification Random Forests for y with predictor importance. ...
0
votes
0answers
88 views

Estimating Bias in a Linear Regression Model

Is there any way to estimate the bias of the estimate of the betas in a linear regression model when the actual beta values are unknown? The well known Mean Square Error (MSE) criterion is used to ...