0
$\begingroup$

Suppose I draw $n$ samples of some random variable $X$. I repeat this process $k$ times so that I end up with $k \times n$ observations.

Each time I draw a random sample, my data is censored, meaning that I cannot observe values larger than some maximum value. Specifically, the values observed is bounded from above by $c_i$ for $i \in 1,…,k$, each time a sample is drawn, where $c_1$, $c_2$, … $c_k$ can be any positive real value.

We know that $X$ follows a lognormal distribution with parameters $\mu$ and $\sigma$.

How do I estimate $\mu$ and $\sigma$ with this censored data, with arbitrary upper bounds $\{c_i\}_{i=1}^k$?

I looked at this source about censored data, but its prescriptions are not clear, and I cannot access the source they cite:

If all observations are censored, the MLE is undefined when there is a single censoring limit and defined but extremely poorly estimated when there are multiple censoring limits. The best approach for such data is to report the median, calculated as the median of the censoring levels (Helsel 2012, p. 143-4)

My intuition says that you can use information about how the estimated moments of the samples related to the limit $c_i$. Suppose the distribution of $X$ is a truncated lognormal and that we know that the support of $X$ is $[a,b]$. Then, as $c_i \rightarrow b$ we should expect the sample moments to be unbiased estimates of the population moments.

Hence, one thing I might try is the following. I could generate estimates for each $i$:

$$\{\hat{\mu}_i, \hat{\sigma}_i\, c_i\}_{i=1}^k$$

I could then model this as follows:

$$\hat{\mu}_i = \alpha_0 + \alpha_1 c_i $$ $$\hat{\sigma}_i = \beta_0 + \beta_1 c_i $$

which I could estimate using, say, OLS. Knowing the support of $X$, I can predict what the value of $\hat{\mu}_i$ and $\hat{\sigma}_i$ would be if $c_i \rightarrow b$. That is:

$$\underset{c_i \rightarrow b}{lim}[\hat{\mu}_i]= \hat{\alpha_0} + \hat{\alpha_1} b$$ $$\underset{c_i \rightarrow b}{lim}[\hat{\sigma}_i]= \hat{\beta_0} + \hat{\beta_1} b$$

Then, if $\hat{\mu}$ or $\hat{\sigma}$ relates to $c_i$ in this linear fashion, this should give unbiased estimates of the population moments. Of course, there's no reason to expect this linear relationship, but I don't currently have other ideas.

Improve this question
$\endgroup$
4
  • 1
    $\begingroup$ If you're trying to say all your observations are censored, then the quotation is accurate: not only is the MLE undefined, all you can possibly hope for is a complex bound on the possible combinations of $(\mu,\sigma).$ $\endgroup$
    – whuber ♦
    Commented Oct 7, 2022 at 1:37
  • $\begingroup$ @whuber yes, all observations are censored, though with different bounds for some subsets of samples. Can you say more about the bounds on $(\mu, \sigma)$? $\endgroup$
    – Tamay
    Commented Oct 7, 2022 at 1:57
  • 2
    $\begingroup$ @Tamay I don't think all your observations are censored (in the usual sense of this term). You seem to be saying that you observe $X_{ij}$ in cases where $X_{ij}<c_i$, otherwise a censoring event is recorded. Presumably the $c_i$'s are known constants(?) in which case you can use standard methods from survial analysis, such as the survreg function in the survival R-package. $\endgroup$
    – Jarle Tufto
    Commented Oct 7, 2022 at 9:42
  • $\begingroup$ @JarleTufto yes, $c_i$’s are known. Could you elaborate on how I could use survival analysis to estimate the moments? $\endgroup$
    – Tamay
    Commented Oct 7, 2022 at 13:30

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.