Testing the estimation of a parameter

I want to estimate the parameters $\theta$ of a distribution, from some data $D = \{x_i\}_{i\leq n}$.

Suppose the result of this estimation is $\theta^*$. I want to test the hypothesis that the data $D$ was sampled from the distribution with parameter $\theta^*$, and in particular, I want to compute the $p$-value of this hypothesis.

I will use a certain statistic $\tau$ in order to do that. $\tau_D$ is the statistic measured on the data, and I need the distribution of $\tau$, $P(\tau)$. In the most general case, a way of getting the distribution is by sampling from the null hypothesis data $S$, and computing $\tau$ for each sample, thus computing an empirical distribution $P(\tau_S)$. Then the one-tailed $p$-value will be $P(\tau>\tau_D)$.

If we agree on this procedure, my question is: can the test statistic $\tau$ can be dependent on the parameter $\theta_*$?

Example: assume that the true distribution of $X$ is $N(0,1)$, and, knowing that the variance is 1, I want to estimate $\mu$. I will use the chi squared as the test statistic, $$\chi^2=\sum_{i=1}^n (x-\langle x\rangle)^2$$ Then I would sample from the distribution $N(\mu, 1)$; in each sample, should I use $$\tau_1=\sum_{i=1}^n (x-\langle x\rangle)^2$$ or $$\tau_2=\sum_{i=1}^n (x-\mu)^2$$ to compute $P(\tau)$ and compare it to $\tau_D$? Notice that $\tau_{1,D}$ and $\tau_{2,D}$ are equal if the estimator of $\mu$ is the mean.

What I know is that $\tau_1\sim\chi^2_{k-1}$ while $\tau_1\sim\chi^2_{k}$, but how to choose generally?

Additionally, for this example, how should I compute the distribution of $p$-values? should I sample from $N(\mu,1)$ or from $N(0,1)$, i.e. which should be the null hypothesis?

Note: I'm not interested in asymptotic behaviour.

• Data is something you observe. So what is null hypothesis data $S$? The way you describe your hypothesis, i.e. $D$ is generated from distribution with parameter $\theta^*$, your null hypothesis data is $D$. So what is $S$? – mpiktas Sep 11 '15 at 10:54
• S is just the name of a sample of data generated from the null hypothesis D; sorry for the confusion – chuse Oct 8 '15 at 9:45
• I want to make sure I've understood correctly. Is τ1 testing the alternative hypothesis that the mean of your data is its mean? And τ2 testing that the mean of your data is the expected mean μ? What exactly is the test for, θ∗ or for the distribution type? – ReneBt Sep 28 '18 at 4:16