4
$\begingroup$

Suppose I have a random variable $X$ ~ $Unif(0,\theta)$ where I want to estimate $\theta$. I draw a sample $X_1,...,X_n$.One way is to get a point estimate using e.g. maximum likelihood estimation via the sufficient statistic $X_{(n)}$.

However, suppose I want a 95% confidence interval of sorts on the value of $\theta$, how should I construct one?

The reason why I am interested in confidence interval is because of the following issue, which is actually my main question (the above prelude on confidence interval is just a suggestion on possible approach). So actually my sample came with some noise, in a sense that I actually observe $X_1+E_1,...,X_n+E_n$ where $E_i$~$N(0,\sigma_i^2)$ independently from each other and from the $X$'s. How should I make some form of statistical estimation about $\theta$? I would prefer some form of confidence estimate.

$\endgroup$
5
$\begingroup$

You can just use maximum likelihood! First, you give the standard deviation of your normal random variable $E_i$ as $\sigma_i$. If that is really what you want, you are more or less out of luck, since you then have more parameters than observations. So I will presuma that was a slip, and assume identical standard deviations $\sigma$.

Then, each of your observations have the distribution of the sum independent uniform and normal variables. So first we must find that distribution, which is an exercise in convolution. Let $f(x) = \frac{1}{\theta}I_{0 \le x \le \theta}(x)$ be the uniform density and $g(e)$ be the normal density. Calculating the convolution we get $$ \begin{align} f \ast g (x) &= \int f(x-e) \frac{1}{\sqrt{2\pi \sigma^2}} e^{-\frac12(\frac{e}{\sigma})^2} \, de = \\ &= \frac{1}{\theta} \int_{x-\theta}^x \frac{1}{\sqrt{2\pi}\sigma} e^{-\frac12(\frac{e}{\sigma})^2} \, de \\ &= \frac{1}{\theta} \int_{\frac{x-\theta}{\sigma}}^{\frac{x}{\sigma}} \frac{1}{\sqrt{2\pi}} e^{-\frac12 u^2} \, du = \\ &= \frac{1}{\theta} \left( \Phi(x/\sigma) - \Phi((x-\theta)/\sigma) \right) \end{align} $$ where we have used a simple substitution, and where $\Phi$ is the standard normal cumulative distribution function. Now, this can be used as usual for maximum likelihood estimation. Let us write the density function we found above as $h(u; \theta, \sigma)$. Then the likelihood function is $$ L(u;\theta, \sigma) = \prod_{i=1}^n h(u_i;\theta,\sigma) $$ so the log-likelihood becomes $$ l(u;\theta,\sigma) = -n\log\theta + \sum_{i=1}^n \log \left ( \Phi(u_i/\sigma) -\Phi((u_i-\theta)/\sigma) \right ) $$ Maximize this over the unknown parameters!

EDIT after the comments: Then I will continue assuming $\sigma$ is known! You can extend it to your case with distinct known $\sigma_i$. We can make a confidence interval on $\theta$ by likelihood profiling. We just use the (asymptotic) result that $$ 2(l(\hat{\theta})-l(\theta)) \sim \chi^2_d $$ where $\sim$ is read "distributed as" and $d$ is the dimension of $\theta$, in our case 1. $\hat{\theta}$ is the maximum likelihood estimator.

You can invert this (numerically) to get a confidence interval.

$\endgroup$
  • $\begingroup$ Opps, I should be clear that all the $\sigma_i$ are actually known. Anyway, I am aware of how to use MLE, I was hoping for a confidence interval kind of method. $\endgroup$ – renrenthehamster Jun 19 '14 at 15:04
  • $\begingroup$ You can make a confidence interval based on the likelihood! for example, based on likelihood profiling. What I have written above can be easily adjusted for known $\sigma_i$. $\endgroup$ – kjetil b halvorsen Jun 19 '14 at 15:06
  • $\begingroup$ I am reading your edits. When you suggest using the asymptotic result, is that Wilk's theorem? I am not too sure if Wilk's theorem is valid here because one of the assumptions of Wilk's theorem is that the support doesn't depend on the parameter. $\endgroup$ – renrenthehamster Jun 20 '14 at 8:23
  • $\begingroup$ In the modified model the support has been "smoothed out" to the whole real line! so that is not a problem! $\endgroup$ – kjetil b halvorsen Jun 20 '14 at 9:29
  • 1
    $\begingroup$ (+1) Caution: for small datasets and many of the $\sigma$ large compared to $\theta$, it's reasonably likely that many of the $x_i$ will be negative. In such cases the likelihood will be maximized at $\theta=0$, where these "asymptotic results" will be incorrect. $\endgroup$ – whuber Jun 5 '17 at 23:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.