The difference between the expected value of a parameter estimator & the true value of the parameter. Do NOT use this tag to refer to the [bias-term] / [bias-node] (ie the [intercept]).

learn more… | top users | synonyms

2
votes
2answers
29 views

Bias of moment estimator of lognormal distribution

I am doing some numerical experiment that consists in sampling a lognormal distribution $X\sim\mathcal{LN}(\mu, \sigma)$, and trying to estimate the moments $\mathbb{E}[X^n]$ by two methods: Looking ...
2
votes
1answer
14 views

Consistent estimation with observed values lower than actual values

Assume an IID sample of the form $ \left\{ y^{r}_{i},\mathbf{x}_{i} \right\}$ (notice the superscript on $y$). The observed values $y^{r}_{i}$ are bounded from above by the actual, unobserved values ...
0
votes
0answers
17 views

Principal Coordinates Analysis (PCoA) with Longitudinal Data

I am interested in running a PCoA on a distance matrix derived from longitudinal data. I'm concerned about biasing the PCoA towards overrepresented subjects (those with more time-points and samples). ...
0
votes
1answer
18 views

Autoencoder with tied weights: bias?

For some unsupervised learning problem, I need to train an autoencoder, so that I only have to store the encoder afterwards. However, I am not sure on how and if the bias weights can be tied. To make ...
0
votes
1answer
29 views

Finding the MSE Using the Delta Method

I don't get the step in the solution for b) can someone please fill in the missing steps between going from eqn (1) to the solution. Thanks. Question: Solution:
1
vote
1answer
30 views

Does increasing sample size have any effect on omitted variable bias?

Say I have a multiple linear regression model, where two of the variables are positively correlated, and I omit one of these variables from the model. First question - if I increase the sample size, ...
3
votes
4answers
149 views

Something wrong with my implementation of the bias/variance diagnostic in polynomial regression

I'm trying to diagnosing bias/variance so I have the below Octavecode: ...
4
votes
1answer
31 views

Why is are unbiased statistics used more commonly than statistics with lower MSE?

I understand the difference between consistency and bias; one converges as the sample size increases, and the other converges as the number of estimates increases, respectively. But, I don't ...
0
votes
0answers
9 views

When on the same data and model an unbiased estimator and a biased estimator give similar values

There was a general consensus here that statements like I calculated Observed $R^2$ and Adjusted $R^2$, and they were pretty similar, suggesting only a small amount of bias in the Observed $R^2$ ...
0
votes
0answers
48 views

Heteroscedasticity and bias shown in residual plots, lme

I have been fitting a linear mixed-effect model. The residual plots are not desirable. I have found many posts telling me the first is heteroscedastic, and the second is biased. But I can't find info ...
2
votes
0answers
21 views

Bias of sample correlation for discrete distributions

Is there a proof showing the bias (or lack thereof) of the sample Pearson's correlation for discrete interval variables? In particular, I am interested in how such a proof handles the expected value ...
0
votes
1answer
56 views

Overspecification bias/ including too many variables to a regression model

This seems to be the general view in statistics community: If the regression model is overspecified (outcome 4), then the regression equation contains one or more redundant predictor ...
0
votes
0answers
8 views

UMVUE for the sample version?

When we say in terms of PARAMETER, there does not exist UMVUE. But what about for the sample? That is, uniformly minimizing MSE$(\frac{1}{n}\sum{(\hat{y}-y_i)^{2}})$ estimator $\hat{y}$ such that ...
0
votes
0answers
13 views

In Random Forest, why IncNodePurity is biased?

I've seen this statement many times, however, I could not find an explicit demonstration of why IncNodePurity biased (actually, how does one define theoretical value of importance, is not so clear, ...
0
votes
0answers
23 views

Suppressor Effects when x1 and x2 are uncorrelated?

I've found this very comprehensive Thread about "Suppressor Effects when x1 and x2 are correlated" and the read the literature that was listed. As far as I understand, a suppressor effect can occur ...
2
votes
1answer
63 views

what is bias and variance of an estimator?

I know what Variance is. But what is Bias? I just have problems to understand this what is written!
1
vote
1answer
51 views

Bivariate probit model with sample selection

Could you please provide an example and explanation why to use the bivariate probit model with sample selection? In this context, to what sample selection bias refers to?
24
votes
2answers
958 views

When is a biased estimator preferable to unbiased one?

It's obvious many times why one prefers an unbiased estimator. But, are there any circumstances under which we might actually prefer a biased estimator over an unbiased one?
0
votes
0answers
25 views

Understanding multicollinearity and bias in coefficient estimates

In trying to better understand the effects of multicollinearity within the context of logistic regression, I have come across the following quote from Paul Allison's textbook: Although ...
0
votes
0answers
14 views

Cancelling roots in ARMA(1,1) with external regressors

I am trying to find out what cancelling roots would imply for the estimators of my external regressors in my ARMA(1,1) model. Unfortunately however I'm stuck in my final step since I'm insecure about ...
1
vote
0answers
16 views

Computing bias due to measurement error:

I'm currently doing an exercise and faced with the following question: We have the true form: $y_i=\beta_0 +\beta_1 d_i +u_i $ Where $d_i$ is a dummy variable. We have measured $d_i$ with ...
2
votes
0answers
19 views

Regression models with several samples of the same people

I am trying to implement some regression models with my data of 56 features x 1500 samples to fit a response variable of 1x1500 and I am hesitating about the statistical validity of what I am trying ...
0
votes
0answers
7 views

Correcting for season-length bias of Gini on win percentage

I made this plot to try and compare the competitiveness of the major US sports (NHL/NBA/MLB/NFL): Each point of a given color represents, for a given season, the Gini coefficient of the win ...
0
votes
0answers
16 views

Bias and precision in estimating species richness?

I would like to estimate the total number of unique species within a community (=richness, for instance two species: cat and dog) from a randomly drawn sample. I am wondering what is the sampling ...
1
vote
0answers
32 views

unbiased estimate for household size

We have a population of N people. They live in households of varying sizes: 1, 2, 3, 4, etc. We are going to do a random telephone survey and ask them how big their household size is. What are the ...
0
votes
0answers
19 views

Notation for computing MSE confuses me?

I wish to compute MSE of my models. Say my data was generated from the following model: $y_i=f(x_i)+e_i$ where $e_i$ is some noise around the true relationship ...
8
votes
2answers
148 views

Is bias a property of the estimator, or of particular estimates?

As an example, I often encounter students who know that Observed $R^2$ is a biased estimator of Population $R^2$. Then, when writing up their reports, they say things like: "I calculated Observed ...
2
votes
1answer
69 views

Omitted variable bias and the constant term

For omitted variable bias to occur when a variable is left out of a regression, there is one axiom and one condition that must be fulfilled: (Axiom) By definition, the coefficient of the variable ...
0
votes
1answer
72 views

Proving that $y_t = \beta_1 x_t + \beta_2 y_{t-1} + u_t$ parameters are biased when $u_t$ is autocorrelated

How do you prove the result that for equation: $$y_t = \beta_1 x_t + \beta_2 y_{t-1} + u_t$$ the beta parameters are biased when $u_t$ is autocorrelated? In other words, that$$ \text{Cov}(u_t, ...
8
votes
1answer
330 views

Are tree estimators ALWAYS biased?

I'm doing a homework on Decision Trees, and one of the questions I have to answer is "Why are estimators built out of trees biased, and how does bagging help reduce their variance?". Now, I know that ...
0
votes
0answers
36 views

Why is bias “constant” in bias variance tradeoff derivation?

I know there are plenty of questions about the Bias/Variance tradeoff. I've been trying to derive it myself to build some intuition. I looked at the Wikipedia page, and I saw this: Notice where ...
0
votes
1answer
64 views

Is there an approximate correction for bias in correlating probability distribution estimates?

I'm computing the correlation between two probability distributions $P(x)$ and $Q(x)$ that I am measuring empirically. Call the estimates $S(x)$ and $T(x)$. The data is binned, so the estimates of the ...
1
vote
1answer
44 views

Controlling for biased audiences in online surveys

For a long time I have been meaning to set up a bunch of online surveys asking a whole range of social questions and publish the results. I am well aware that there are various difficulties to ...
10
votes
3answers
659 views

Do we really need to include “all relevant predictors?”

A basic assumption of using regression models for inference is that "all relevant predictors" have been included in the prediction equation. The rationale is that failure to include an important ...
0
votes
0answers
11 views

Bias and Expected Value of Residuals

If I am running an OLS regression is it always the case that the residuals sum to 0? Or, if there is a bias, for example, their exists endogeneity, then when I run the OLS regression the sum of the ...
1
vote
1answer
27 views

Role of the bias term in regression

I was trying to understand the role of the bias term in linear regression which is given by, y=w^T. phi(x)+b From what I understand it allows for any fixed ...
0
votes
0answers
10 views

Ratio of TPR and TNR as a measure of biasedness?

Can the ratio of true positive rate (TPR) and true negative rate (TNR) be considered as a measure of biasedness? For example, classifier a has a TPR of 0.6 and a TNR of 0.4, while classifier b has a ...
0
votes
0answers
13 views

How do I evaluate the impact of sample bias on the generalizability of my findings?

I am performing a regression analysis to identify predictors for my DV, using survey data. When comparing auxiliary information on my participants (in my case firm size and industry) to the available ...
0
votes
1answer
33 views

Understanding the bias variance trade-off of the regression function

If we take $\mu$ to be the true regression function, and we estimate $\mu$ by $\hat\mu$ from the available data, which is random, and therefore so is the estimate, which we may denote by $\hat M_n$ to ...
1
vote
2answers
75 views

Why isn't variance considered a bigger deal than bias?

In statistical literature, people say: Let's avoid making a biased estimator, that will mess things up! Why not avoid also having a varianced estimator? They should be treated the same.
4
votes
1answer
50 views

Are there two definitions of the word bias?

I hear the term bias being thrown around a lot in statistical literature. For example, By using mean-wise imputation, we are adding bias to our estimate. Another example, The ...
4
votes
0answers
35 views

Decomposition of average squared bias (in Elements of Statistical Learning)

I can't figure out how formula 7.14 on page 224 of The Elements of Statistical Learning is derived. Can anyone help me figure it out? $$\textrm{Average squared bias} = ...
1
vote
0answers
16 views

Solving for Bias in Simultaneous Equations Model without Instruments

Suppose you have the following structural equations where wage and status are determined simultaneously and you do not have any instruments for wage or status: $(1) \text{wage}_i = \alpha_w + ...
0
votes
0answers
22 views

Can PCA be used to reduce estimation error for covariance estimation?

One of the uses of factor models is to estimate covariance matrices. The reason why you might want to do this is to reduce the number of variables that you have to estimate and so avoid accumulating ...
0
votes
1answer
22 views

How to match a biased sample to a population?

I have a sample of people which is biased in age, gender, geography. I am trying to measure various continuous outcomes out of them. I have the census data to tell me the reality of the population ...
1
vote
1answer
45 views

Derive bias when AR(1) is approximated by MA(1)

Consider the MA(1) process: $$ y_t = \varepsilon_t + \theta_1 \varepsilon_{t-1} $$ where $\varepsilon$ is a white noise process with $\mathbb{E}(\varepsilon_t) = 0$ and ...
1
vote
0answers
31 views

Bias in the regression coefficients of a generalized linear model under MLE

Question: Are the regression coefficients of a generalized linear model biased when estimated through maximum likelihood? Imagine, we have a generalized linear model where $E[Y] = g^{-1}(\mu)$ for ...
0
votes
1answer
49 views

How does CausalImpact Prevent Overfitting

I'm using Google Research's Causal Impact package, and I'd like to understand more fully how the package prevents overfitting and selecting a bad batch of the covariates by chance. Here's the ...
0
votes
0answers
24 views

Truncated dependent variable caused by the researcher!

I created my own index for the study on firms. I am interested in using the index as the dependent variable. According to other statisticians, firms with less than 100 observations used to create the ...
0
votes
0answers
17 views

What is the trade-off in the bias-variance trade-off? [duplicate]

Let $\theta$ be a parameter and $\hat{\theta}$ be an estimator for $\theta$. I understand that the MSE of $\hat{\theta}$ can be decomposed into its bias and variance. That makes sense. What I don't ...