0
$\begingroup$

I'm interested in a situation where I have a small list of events, somewhere in the range ten to a few hundred.

  1. Each event can be a 1 or 0 outcome.
  2. I have a good reason to believe that the factors controlling the 1 or 0 outcome are complicated and a "1" does not necessarily occur a constant rate.
  3. I already know the outcome of all events in the list to date.
  4. There will be future events whose outcome is unknowable at present.

I want to use all the information available to help me estimate the rate of "event outcomes = 1", especially for the next event to occur. I actually also have a number of lists A, B, C etc, each where I am interested to understand how similar or different their rates are.

I've looked at these questions and their answers:
statistical-inference-when-the-sample-is-the-population

what-is-the-difference-between-a-population-and-a-sample

Help needed with terminology

I had originally assumed that the current list of events that have taken place is the "sample" and that the "population" refers to some larger group, that could be the current list + 1, or more. I don't mind how we define it as long as we don't lose sight of the practical human language and meaning that I need to operate in the real world.

I am interested in the next event specifically, and probably won't become interested in the one after that until the next has occurred.

Discussion

I follow that if population = sample then there is no role for statistical tests. I think that perhaps population = sample + 1 because I am interested in the next event. However if the sample is almost the same size as the population then there is still little scope for the statistic to vary from sample to sample, especially if the sample cannot be held to be a random sample of the population.

Question: To establish some measure of belief in the rate of event outcomes, the proportion p of population n, should I use a measure such as

  1. $$ \pm \sqrt{ \frac{p ( 1-p)} {n}} $$

  2. some other measure based on the changes to p that would occur upon re-evaluation after then next future event, should it turn out to be either a 0 or a 1.

$\endgroup$
5
  • $\begingroup$ "There will be future events whose outcome is unknowable at present" tells us you don't have the entirety of the relevant population, which evidently includes all those future events. Please tell us, then, exactly what you mean by "population," because you don't seem to be using its standard meaning. $\endgroup$
    – whuber
    Commented Apr 9, 2019 at 19:26
  • $\begingroup$ @whuber Thank you, this is a step towards the kind of help I need. I'm not a statistician so please forgive the terminology problems. I've replaced "population" with "list" a few times early on so that it doesn't have a reserved meaning. $\endgroup$
    – Puffin
    Commented Apr 9, 2019 at 19:40
  • $\begingroup$ to understand what is sample one needs to understand what is event $\endgroup$
    – Aksakal
    Commented Apr 9, 2019 at 20:44
  • $\begingroup$ Its the thing we are counting with a view to performing statistical tests. Other events could be throws of a dice (its not), or balls pulled out of a sack (its not this either). An event is something that, when it happens, has an outcome of 1 or 0. Ten events, or a hundred events, could be a sample size of ten or a hundred respectively. $\endgroup$
    – Puffin
    Commented Apr 9, 2019 at 21:12
  • $\begingroup$ Your item (2) may be a deal-breaker. If you really mean that $p$ varies from trial to trial in a way you don't understand, then it will be difficult to get a probability model for the process. If you mean that "0" or "1" can occur on any one trial for reasons you don't understand, but you believe $p$ (probability of getting a "1") is the same throughout, then maybe you're dealing with a binomial model. $\endgroup$
    – BruceET
    Commented Apr 9, 2019 at 21:54

1 Answer 1

3
$\begingroup$

I suspect that your events are not independent but rather involve some sequential dependencies, because you mentioned that the rate is not constant and don't know any controlling variables. The sample is the portion of the data randomly sampled from the population. Since you use a deterministic range to define your sample (e.g. select first N events out of the list of N+1 events) you can not treat it as a valid sample.

Thankfully, as you already noted, you can compute your population statistics directly. Yet those statistics wouldn't lead to any reasonable outcomes, because you've also noted that the rate is not constant - and those statistics can provide you only with a constant.

You can approach the prediction problem in multiple ways. I'll provide you one just to give you a hint. Your problem is actually to model a probability $p(e_t = 1 | e_{t-1}, e_{t-2}, ..., e_0)$ of observing the event at time $t$ given the information about all previous events. It is often the case that it is not necessary to consider all of the previous events, but only limit yourself the last K. You end up with $p(e_t = 1 | e_{t-1}, e_{t-2}, ..., e_{t-K})$. For simplicity you could just assume, that this probability equals to the rate of the events within this K-length window:

$$p(e_t = 1 | e_{t-1}, e_{t-2}, ..., e_{t-K}) = \frac{1}{K}\sum_i^K e_{t-i}$$

You can then evaluate this model on different windows sampled from your list in order to find the best window size. You can then improve this model by introducing new assumptions and/or parameters. Take a look at language models and kernel density estimation.

$\endgroup$
4
  • $\begingroup$ You are right that in my case "It is often the case that it is not necessary to consider all of the previous events,". I have already tried something like this, unless I am mistaken the K-length window is a moving average filter. The choice of K usually leads to several, say 20, 30, 50, 100 though this is as far as I have got, as it appears as much driven by external beliefs about the relevance of one window length or another. $\endgroup$
    – Puffin
    Commented Apr 11, 2019 at 7:56
  • $\begingroup$ is there any measure of uncertainty associated with this window? I can see I could just choose that to be the sample or is that not possible if we are calling the window a deterministic range? $\endgroup$
    – Puffin
    Commented Apr 11, 2019 at 20:06
  • $\begingroup$ What this average tells you is the parameter of the Bernoulli distribution used to generate your data. To assess its uncertainty you can compute the likelihood of your data given this model. You collect random samples (windows) from your data. The probability of each sample is either the rate $r$ if it is followed by an event or $1-r$ if it's not. The products of such probabilities is the overall likelihood. This is going to be a very small value so you'd probably like to use the logarithm of this value (log-likelihood). This should be familiar to the commonly used cross-entropy score. $\endgroup$
    – pkubik
    Commented Apr 12, 2019 at 14:10
  • $\begingroup$ Note that you can use this score to compare two different models (e.g. models with different window size $K$) so you could use an optimization algorithm to find the best model. That means that you would use maximal likelihood estimate (MLE). You can try different families of models, e.g. replace the rectangular window with gaussian kernel in order to make older events less important than the recent ones. $\endgroup$
    – pkubik
    Commented Apr 12, 2019 at 14:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.