# How close to being memoryless can you make a distribution with bounded support?

Related to Exponential-like distribution with support [0,1] I wondered just how close to memorylessness a continuous distribution with bounded support can get. For a continuous variable to be memoryless, it has to be exponential, just as a discrete memoryless distributions must be geometric, so this is a defining feature of the exponential distribution. If the support is bounded, the distribution cannot be exponential so cannot be memoryless, but we may still be able to define a sense in which it comes "close" to being memoryless.

We say a continuous distribution is memoryless if for all $$s, t \geq 0$$ we have $$\Pr(X>t+s \mid X>t)=\Pr(X>s)$$.

Let's say that we have got "close" to being memoryless if, for example, the absolute value of $$\Pr(X>t+s \mid X>t) - \Pr(X>s)$$ is very small for any choice of $$s, t$$ and we might want to restrict it so that $$X, s, t, s+t$$ all lie between 0 and 1. One metric for "closeness to memorylessness" might be the least upper bound for that absolute value of the difference, but if another metric has been proposed before that's fine too.

So whichever sensible way we measure it, just how close to memorylessness can we get?

I suspect the answer is we can get arbitrarily close by using a truncated exponential distribution with mean increasingly nearer to zero. But for a fixed mean of $$X$$, e.g. $$\mathbb{E}(X) = 0.1$$, it's no longer intuitive (at least to me) that a truncated exponential would be optimal... does anyone have any suggestions? Is it something that has been researched?

• – whuber
Sep 18 '20 at 15:44
• @whuber Indeed, I meant to state I wanted a continuous distribution - which I thought I had, but reading through I see that I haven't... will make a quick edit. Sep 18 '20 at 15:49
• @whuber Yes this is what I was referring to at the end of the question, I suspect the question is more interesting if we set the support to be $[0,1]$ then impose a restriction that prevents taking the limit of truncated exponential distribution like that, e.g.. requiring $\mathbb{E}(X)=0.1$, Sep 18 '20 at 15:59
• Agreed, because that constraint eliminates the trivial constructions I was describing.
– whuber
Sep 18 '20 at 16:03
• for the discrete case, a true memoryless bounded distribution is a dirichlet in 1. you get a process where the event always happens, no memory required. Sep 19 '20 at 8:15

In terms of the CDF $$F(t)$$ or the survival function $$S(t) = 1-F(t)$$ you have

$$p(X>t+s|X>t) = \frac{S(t+s)}{S(t)}$$

You get this fraction to be constant for different $$t$$ and $$s$$ when $$S(t)$$ is an exponential function.

(And obviously the relation breaks when $$t>1$$ or $$t+s>1$$, because that exponential relation ends above 1. So you only have memoryless in some narrow sense)

### Truncated exponential with point masses

We can have an exponential function for the survival function as follows

$$S(t) = \begin{cases} 1 &\quad & \text{for t\leq0}\\ a \exp(-bt) &\quad &\text{for 01} \end{cases}$$

This is a truncated exponential distribution with extra point masses at $$t=0$$ and $$t=1$$ (a mixture of a continuous and discrete distribution).

The most extreme case is when you have a single point mass at $$t=1$$, by setting $$a=1$$ and $$b=0$$, which is $$S(t)=1$$ for $$t<1$$ and $$S(t)=0$$ for $$t\geq1$$. Or when you have a single point mass at $$t=0$$, by setting $$a=0$$, in which case the definition of the conditional probability (which equals zero) becomes a vacuous truth.

### Truncated exponential

At first I thought that the truncated exponential would satisfy as well. But in this case the survival function will be

$$S(t) = \begin{cases} 1 &\quad & \text{for t\leq0}\\ \frac{\exp(-bt) - \exp(-b)}{1-\exp(-b)} &\quad &\text{for 01} \end{cases}$$

It is translated/shifted by a constant $$\exp(-b)$$ to ensure that $$S(1)=0$$ and continuous.

### If the distribution needs to be continuous

In this case you can use the distribution with point masses and substitute the point masses with a continuous function and make them arbitrarily small.

You can also use the truncated exponential and make the constant $$\exp(-b)$$ arbitrarily small. In the extreme cases $$b\to \infty$$ you approach the situation with a point mass in $$t=0$$.

• (+1) Hadn't thought about adding a point mass in, nice idea! Still leaves me wondering just how good you can get with a continuous distribution... Sep 18 '20 at 19:38
• One idea is to follow stats.stackexchange.com/questions/188903/… and minimize a KL divergence, the solution to the minimization problem is obvious (but that will not give point masses) Sep 18 '20 at 19:39
• @Silverfish, I believe I made a mistake with the truncated exponential. It's survival function is not a plain exponential. Sep 19 '20 at 7:23