Bias and expected value of estimator

Question

Definition from Wikipedia:

$\operatorname {Bias} _{\theta }[\,{\hat {\theta }}\,]=\operatorname {E} _{\theta }[\,{\hat {\theta }}\,]-\theta =\operatorname {E} _{\theta }[\,{\hat {\theta }}-\theta \,],$ where ${\displaystyle \operatorname {E} _{\theta }} $ denotes expected value over the distribution ${\displaystyle P(x\mid \theta )}$, i.e. averaging over all possible observations ${\displaystyle x}$ . The second equation follows since θ is measurable with respect to the conditional distribution ${\displaystyle P(x\mid \theta )} .$

Is this a general definition of bias? Because here, we are conditioning on $\theta$, so this implies that $\theta$ is a random variable, whereas in frequentist approach, we assume that the true parameter $\theta$ is fixed but unknown, so $P(x; \theta)$

Second definition I came across somewhere:

where the expectation of estimator is over the data (seen as samples from a random variable)

What does it mean in this context that expectation of estimator is over data? Could you provide exact formula for $\operatorname {E} _{\theta }[\,{\hat {\theta }}\,]$?

Answer 1

Definition from Wikipedia seems clear to me.

An estimator $\hat{\theta}$ is simply a function of observed data $X$, i.e. a statistic. We have $\hat{\theta} = g(X)$.

Bias is taken w.r.t. conditional distribution of $X$ on specific $\theta$, so

$$E_{\theta}[\hat{\theta}] = E_{\theta}[g(X)] = \int_{X} g(x) p(x|\theta) dx, \text{ for some } \theta \in \Theta $$

To elaborate a bit, we say that an estimator $\hat{\theta}$ of $\theta$ is unbiased if

$$E_{\theta}[\hat{\theta} - \theta] = 0,\ \text{for every } \theta \in \Theta$$