4
$\begingroup$

I am interested in predicting a normal distribution, but not sure if this is possible.

I do not have information on the mean or standard deviation. However, I know the range of values, let's say from 0 to 10, and I know the sample size, let's say 1000, and I also know that value 8.1 is the 50th highest value.

Is it possible in any way from this information to infer more about what the distribution looks like?

$\endgroup$
12
  • $\begingroup$ "However, I know the range of values" ... Do you mean by this that you know the highest and lowest values in the sample? $\endgroup$
    – Silverfish
    Commented Apr 14, 2016 at 21:54
  • $\begingroup$ Yes, sorry that is what I meant, they range from 0 to 10. So 0 is lowest and 10 is higher. $\endgroup$
    – AdrianP.
    Commented Apr 14, 2016 at 21:55
  • 1
    $\begingroup$ It sounds to me like you know three "order statistics" for your sample (the 1st, 50th and 1000th/last) and want to estimate (think that's more appropriate than "predict") the parameters of your distribution, is this correct? If not then feel free to revert my edit, better (clearer) titles generally attract answers! $\endgroup$
    – Silverfish
    Commented Apr 14, 2016 at 21:55
  • $\begingroup$ Yes! that makes sense. $\endgroup$
    – AdrianP.
    Commented Apr 14, 2016 at 21:58
  • 1
    $\begingroup$ Do you really mean 50th, or do you mean 50th%, i.e., 500th (or 501)? $\endgroup$ Commented Apr 14, 2016 at 22:00

2 Answers 2

10
$\begingroup$

Use maximum likelihood.

Generally, suppose you have sorted array of order statistics $x_{[i]}$ for $i$ in a subset $\mathcal{I}$ of $\{1,2,\ldots, n\}$. Augment this vector to include $x_{[0]}=-\infty$ and $x_{[n+1]}=\infty$. You are supposing the underlying distribution is $F_{\theta}$ with corresponding density $f_{\theta}$ and you wish to estimate $\theta$. The likelihood of these order statistics, up to a factor that will not vary with $\theta$, is a product of the $f_\theta{x_{[i]}}$ (excluding $x_{[0]}$ and $x_{[n+1]}$, where necessarily $f=0$) times all the powers

$$\left(F_\theta(x_{[i_{j+1}]}) - F_\theta(x_{[i_j]})\right)^{i_{j+1}-i_j-1}$$

for $j$ from $1$ onward. In the example of the question this would be

$$f_\theta(0)f_\theta(8.1)f_\theta(10)\ \left(F_\theta(8.1) - F_\theta(0)\right)^{949}\left(F_\theta(10) - F_\theta(8.1)\right)^{48}.$$

The rest is completely routine, provided you have at least as many order statistics as there are components of $\theta$: there typically will be a unique value of $\theta$ that minimizes this expression. In the question, $\theta$ has two components and there are three order statistics, so all is fine. The machinery of ML produces, in a standard way, estimated standard errors of the parameters, too.


To illustrate, the R code below estimates $\hat\mu=5.39794$ and $\hat\sigma=1.62553$ from the data in the question (namely, $(x_{[1]}, x_{[951]}, x_{[1000]}) = (0, 8.1, 10)$, with $n=1000$). Then, as a quick visual check that the estimates are reasonable, it generates $1000$ datasets of size $n=1000$ from this Normal distribution, records the $1,951,$ and $1000$ order statistics (as given in the question), plots their histograms, and superimposes the observed order statistics on those histograms. The fit is beautiful for orders $1$ and $951$ and reasonable for order $1000$ (the value of $10$ is around the twelfth percentile of this distribution--not too extreme).

Figure of three histograms

#
# Negative log likelihood.
#
Lambda <- function(theta) {
  mu <- theta[1]
  log.sigma <- theta[2]
  sigma <- exp(log.sigma)
  f <- dnorm(values, mu, sigma, log=TRUE)
  F <- log(diff(pnorm(c(-Inf, values, Inf), mu, sigma)))
  -(sum(f) + sum(F * (diff(c(0, orders, n+1))-1)))
}
#
# The data.
#
n <- 1000
orders <- c(1, 951, 1000)
values <- c(0, 8.1, 10)
#
# Compute the estimate.
#
theta.start <- c(mu=mean(range(values)), log.sigma=log(diff(range(values))/6))
fit <- nlm(Lambda, theta.start) # It converged.
theta.hat <- fit$estimate
mu.hat <- theta.hat[1]
sigma.hat <- exp(theta.hat[2])
#
# Check the quality of the estimate visually.
#
n.sim <- 1e3
n <- 1000
sim <- apply(matrix(rnorm(n.sim*n, mu.hat, sigma.hat), n), 2, sort)[orders, ]
par(mfrow=c(1, length(orders)))
invisible(
  sapply(1:length(orders), function(i) {
    hist(sim[i,], freq=FALSE, xlab="Value", main=paste("Order", orders[i]))
    abline(v=values[i], lwd=2, col="Red")
  })
)
$\endgroup$
0
-1
$\begingroup$

You have a sample of random numbers $x_1,x_2,\dots,x_{1000}$. What's know is $\min[x_i],\max[x_i]$ and 50th highest number A, which is 5th percentile in a sample 1000 long.

Given the information I'd go with $$\hat\mu=\frac{\min+\max}{2}=5$$, and $$\hat\sigma=\frac{A-\hat\mu}{1.65}=\frac{8.1-5}{1.65}\approx 1.9$$

The t-statistics 1.65 is derived from 5% one tail z-value. I bet that you can come up with a rigorous MLE procedure which would produce the same result

UPDATE

Here's another way of estimating the dispersion using @jth idea. The max and min upper bounds are: $$\max=\mu+\sigma(\sqrt{2\log n} +1)$$ Hence, $$\hat\sigma=\frac{\max-\min}{2(\sqrt{2\log n} +1)}=\frac{10}{2(\sqrt{2\log 1000}+1)}\approx 1.4$$

$\endgroup$
8
  • $\begingroup$ The max of $n$ Gaussians scales like $\sqrt{2 \log n}$. So you'd want to renormalize $\hat{\mu}$. $\endgroup$
    – nth
    Commented Apr 27, 2017 at 18:20
  • $\begingroup$ @jth Gaussian's a symmetric distribution, I can't see how max could have different dynamics than min. The center must be right in between min and max $\endgroup$
    – Aksakal
    Commented Apr 27, 2017 at 18:31
  • 3
    $\begingroup$ This isn't a great solution. For the example data, we may compare this estimate to the Maximum Likelihood estimate by subtracting the log likelihoods and doubling them to produce their deviance. The deviance is around $1300$, amounting to a huge error. You have substantially overestimated the standard deviation. $\endgroup$
    – whuber
    Commented Apr 27, 2017 at 18:42
  • 1
    $\begingroup$ The new estimator of $\sigma$ completely ignores the third order statistic, using only the extremes of the data in its calculation. As such we can expect it to have a substantially larger variance than other estimators that use all the data. This would make it an inadmissible estimator. There are some nice estimators based on linear combinations of order statistics: they are especially useful with certain kinds of censored data. For a brief description see, e.g., itrcweb.org/gsmc-1/Content/GW%20Stats/…. $\endgroup$
    – whuber
    Commented Apr 27, 2017 at 20:11
  • 1
    $\begingroup$ When you have only three data points, and two of them are as imprecise as possible (the extremes in this case), it's rarely a good idea to ignore the third and most reliable value! The contrast really is stark in this example: the variances of the maximum and minimum are 1.5 orders of magnitude greater than the variance of the fiftieth largest value: compare the spreads in the three histograms in my answer. $\endgroup$
    – whuber
    Commented Apr 28, 2017 at 17:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.