Consider a sample $x_1,\ldots,x_n \sim F$ from an unknown parametric distribution where $F$ is the cumulative distribution. We observe data in the form $F(x_1),\ldots,F(x_n)$. Stated differently, we observe some points on the unknown cumulative distribution function. We are interested in finding the best fitting parametric distribution.

If we assume a functional form for $F$, say a normal distribution $\Phi$, such that $F(x_j) = \Phi(x_j|\theta) + \epsilon_j,\ \epsilon_j \overset{\text{iid}} \sim \mathcal{N}(0,s^2)$, then one can find $\theta = (\mu, \sigma^2)$ by minimizing the sum of squared distance between the observed values and a normal cdf evaluated at values $x_1,\ldots,x_n$

$$ \underset{\theta} \min \sum_{j=1}^n (F(x_j) - \Phi(x_j|\theta))^2 $$

This looks similar to a non-linear least squares problem. However, since the data is sequential, I do not want to optimize again when I get a new observation. Instead, I would like to update the parameters of my normal distribution using a simple update. Is it possible to setup a prior on $\theta$ and get a posterior $p(\theta|x_1,\ldots,x_n)$, then I can use the posterior to get a posterior predictive distribution values for unobserved $x_{n+1}$ $$ \Phi(x_{n+1}|x_1,\ldots,x_n)=\int_{\Theta} \Phi(x_{n+1}|\theta)p(\theta|x_1,\ldots,x_n)d\theta $$

  • 1
    $\begingroup$ Let $y_i = F(x_i|\theta)$. Do you observe $x_{1:n} = (x_1, \ldots, x_n)$ or $y_{1:n} = (y_1, \ldots, y_n)$ or both? $\endgroup$ – mef Aug 9 '18 at 21:12
  • $\begingroup$ Because $x_j$ appears alone in the minimization formula, evidently you have observed ordered pairs $(x_j, F(x_j\mid\theta)).$ In most circumstances, wouldn't that let you solve exactly for $\theta$ after making $n$ independent observations where $n$ is the dimension of $\theta$? E.g., in your Normal distribution example, $n=2$ ought to suffice. What's the point of solving the minimization problem--or indeed of obtaining any more than $n$ observations? $\endgroup$ – whuber Aug 9 '18 at 21:22
  • $\begingroup$ @mef I observe both $\endgroup$ – Rohit Arora Aug 9 '18 at 21:30
  • $\begingroup$ @whuber If $x_i$ are unique then you point is well taken, I can solve a system for both $(\mu, \sigma)$ with just two observations. However, $x_i$ may be repeated and corresponding $y$ values may be different. $\endgroup$ – Rohit Arora Aug 9 '18 at 21:39
  • $\begingroup$ If you are sampling from continuous distributions, then the chance of a repetition is zero. Your formulation is very strange: could you edit your post to explain what you're doing in non-mathematical terms? In particular, could you explain exactly how you observe the values of a CDF and why there are no errors in any of your observations? $\endgroup$ – whuber Aug 9 '18 at 21:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.