4
$\begingroup$

I’m having difficulties dealing with a time series of relations between two numbers.

I have two time series, essentially a count of "successes" and "trials". What I'm interested in, though, is the relation between these two numbers: I want to extrapolate this success rate to the whole population.

Another complication is that I'm sure (as in, 99% sure) that earlier periods are heavily biased and don't represent the population well.

Now, I'm not sure how I could model this. I tried modelling the series as a random walk for the log of the ratio. Something like (following this example: ):

with pm.Model() as model:
    smth_parm = pm.Uniform("smth_parm", lower=0, upper=1)
    tau2 = pm.Exponential("tau", 1.0/(50.0**2.0))
    z2 = pm.GaussianRandomWalk("z2",
                           tau=tau2 / (1.0 - smth_parm), 
                           shape=len(rates_df))
    obs = pm.Normal("obs", 
                    mu=z2*np.log(rate_df.rate_lag),
                    tau=tau2 / smth_parm, 
                    observed=np.log(rates_df.rate))

And got this result (sampling from Posterior Predictive Checks):

enter image description here

I'm PRETTY sure the initial period should have a mean closer to the later ones, even at the cost of an even higher variance. But I can't see how to force that into the model.

So, I need help to:

1- Model this ratio

2- "Force" the model to admit the earlier periods as biased

And I know this whole "Series of Ratios of Successes" stinks like a Beta(Binomial) should be involved, but I'm not sure how. And yes, the number of successes and trials do increase over time, but even considering this, the beginning is biased "up".

Most guides and tutorials are about forecasting, but that’s not what i need. I simply wish to create a model that better describes my data. Specifically, the future better informing past entries

$\endgroup$
  • 1
    $\begingroup$ I wonder if generalized linear autoregressive moving average (GLARMA) modeling might be useful for your problem, as it can handle a binomial time series? You could add a covariate in your model which keeps track of what period you are dealing with (early, late, etc.). See routledgehandbooks.com/pdf/doi/10.1201/b19485-5 and also note that R has a glarma package. $\endgroup$ – Isabella Ghement Aug 18 '18 at 2:16
0
$\begingroup$

Some advice from the font of all time series knowledge ( grin ! )

  1. when you convert two column to one column ... you lose information thus dp not do this
  2. model y as a function of x .. predict x to obtain a prediction of y and then if you are so inclined compute the ratio of the predicted y to the predicted x to obtain a predicted ratio
  3. Encode the uncertainty in the prediction of x into the transfer function model between y and x while
  4. detect unusual activity be it pulses , level shifts, seasonal and/or local time trends . Include the possiblity of future pulses in the prediction limits around the expected value of y

5 detect possible changes in error variance that may have occured within time segments as is the case with your data

  1. verify constancy of model parameters over time suggesting data segmentation

If you wish you can post your data and I will try to be of more help ..

$\endgroup$
  • 2
    $\begingroup$ Could you explain what you mean by "x" and "y" and how they are related to the question? And how does your advice depend, if at all, on the stated desire for the "future better informing past entries"? How do your proposed techniques accommodate the implied Binomial distributions of the successes? $\endgroup$ – whuber Aug 17 '18 at 21:38
  • $\begingroup$ X are trials Y are succeses .. I don't know what "future better informing past entries" means in English... do you ? . The larger the # of trials .. the more "normal" are the # of successes $\endgroup$ – IrishStat Aug 17 '18 at 22:32
  • $\begingroup$ If the # of trials is "small" then perhaps a non-parametric tool might be called for ..... $\endgroup$ – IrishStat Aug 17 '18 at 22:38
  • 1
    $\begingroup$ I'm not quite sure what "future ... past" means either, but (a) it sounds like a non-standard problem and (b) when you're not sure, please post a comment to the question asking for clarification before you attempt an answer. BTW, the plot shows there must be at least tens of thousands of trials in that time series. $\endgroup$ – whuber Aug 18 '18 at 14:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.