12
$\begingroup$

I have data for the population of a number of different fish, sampled over a period of about 5 years, but in a very irregular pattern. Sometimes there are months between samples, sometimes there are several samples in one month. There are also many 0 counts

How to deal with such data?

I can graph it easily enough in R, but the graphs are not particularly illuminating, because they are very bumpy.

In terms of modeling - with species modeled as a function of various things - maybe a mixed model (aka multilevel model).

Any references or ideas welcome

Some details in response to comments

There are about 15 species.

I am trying to both get an idea of any trends or seasonality in each fish, and look at how the species are related to each other (my client originally wanted a simple table of correlations)

The goal is descriptive and analytic, not predictive

Further edits: I did find this paper by K. Rehfield et al., which suggests using Gaussian kernels to estimate the ACF for highly irregular time series

http://www.nonlin-processes-geophys.net/18/389/2011/npg-18-389-2011.pdf

Improve this question
$\endgroup$
6
  • 1
    $\begingroup$ I'm not the right guy to answer your question, but a multilevel model sounds reasonable. Any hints on how large the samples are, how many species there are, and how the zero counts come about? (On the last point, are the samples attempts at random samples, or are they biased, like you just got the counts from a bass-fishing contest which will probably not yield any catfish?) $\endgroup$
    – Wayne
    Commented Aug 16, 2011 at 21:17
  • 1
    $\begingroup$ "Deal with" means what, exactly? For some ideas about coping with irregular times search this site on "+irregular +time" $\endgroup$
    – whuber ♦
    Commented Aug 16, 2011 at 21:22
  • $\begingroup$ Can you clarify the sampling and the goal? For instance is this capture-recapture? Is it a net placed in a stream for a particular period of time, without release? Are you trying to estimate future sample sizes or the larger population from which a sample is drawn? Are the samples from 1 or multiple locations? There's nothing wrong with irregular time series, but it's a little hard to understand the connection between sampling events and between the samples and some target variable (e.g. a model response). Also, is the goal predictive or descriptive in nature? $\endgroup$
    – Iterator
    Commented Aug 16, 2011 at 22:11
  • 2
    $\begingroup$ Why'd someone vote this question down? Why not try to help develop a better question or answer? $\endgroup$
    – Iterator
    Commented Aug 17, 2011 at 4:26
  • 2
    $\begingroup$ @Iterator Because even now, after "further edits," there is no clear question here. The downvote (delivered after no response was observed to my first comment) was placed to encourage the OP to provide the necessary improvements, as well as a signal of the only partially-formed state of the question as it stood. It's not the job of every reader (nor the mods, for that matter) to guess what's intended! $\endgroup$
    – whuber ♦
    Commented Aug 17, 2011 at 20:27

2 Answers 2

Reset to default
12
$\begingroup$

I have spent quite some time building a general framework for unevenly-spaced time series: http://www.eckner.com/research.html

In addition, I have written a paper is about trend and seasonality estimation for unevenly-spaced time series.

I hope you will find the results helpful!

Improve this answer
$\endgroup$
2
  • 5
    $\begingroup$ Thanks! That analysis was long ago and I am no longer doing it, but similar things may come up again; and others do search these threads a lot, so your comment is not wasted. $\endgroup$
    – Peter Flom
    Commented Oct 14, 2012 at 21:53
  • $\begingroup$ Thanks for the information (and indeed years later someone on the internet is looking for it!), but the link has gone dead. $\endgroup$
    – Hooked
    Commented Jun 11, 2014 at 14:27
2
$\begingroup$

I don't know if a mixed model is very appropriate (using the standard packages where the random effect structure is a linear predictor), unless you think the data at all time points should be exchangeable with each other in some sense (in which case the irregular intervals are a non-issue) - it wouldn't really be modeling the temporal autocorrelation in a reasonable way. It's possible you could trick lmer() into doing some sort of autogressive thing but how exactly you'd do that escapes me right now (I may not be thinking straight). Also, I'm not sure what the "grouping variable" would be that induces autocorrelation in the mixed model scenario.

If the temporal autocorrelation is a nuisance parameter and you don't expect it to be too large, then you could bin the data into epochs that are essentially disjoint from each other in terms of correlation (e.g. separate the time series at points where there are months of no data) and view those as independent replicates. You could then do something like an GEE on this modified data set where the "cluster" is defined by which epoch you are in, and the entries of the working correlation matrix are a function of how far apart the observations were made. If your regression function is correct, then you will still get consistent estimates of the regression coefficients, even if the correlation structure is misspecified. This would also allow you to model it as count data using, for example, the log-link (as one usually would in poisson regression). You could also build in some differential correlation between species, where each time point is viewed as a multivariate vector of species counts with some temporally decaying association between time points. This would require some pre-processing to trick the standard GEE packages into doing this.

If the temporal autocorrelation is not a nuisance parameter, I would try something more like a structured covariance model where you view the entire dataset as one observation of a big multivariate vector such that covariance between observations $Y_{s},Y_{t}$ on species $u,v$ is

$$ {\rm cov}(Y_{s}, Y_{t}) = f_{\theta}(s,t,u,v) $$

where $f$ is some parametric function known up to a finite number of parameters, $\theta$, along with a number of parameters to govern the mean structure. You might need to "build your own" for a model like this, but I'd also not be surprised if there are MPLUS packages to do things like this for count data.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Thanks @macro. I think that a mixed model might be OK because they are often used for data that is nested in time; I am not so interested in modeling the autocorrelation - that is, it is a nuisance. I agree time won't be linear, but I can add effects of time (not sure which ones yet, but I can explore it). I don't have MPLUS, but I have R and SAS. $\endgroup$
    – Peter Flom
    Commented Aug 17, 2011 at 13:17
  • 1
    $\begingroup$ I'm only saying that a standard mixed model might not be appropriate in a this situation. The random intercept is useless if you don't think times points are exchangeable in terms of correlation (i.e. it would only offer an approximation within the 'exchangeable correlation' world to your true correlation structure). Including random slopes in time indicates you think that the trajectory is "heading somewhere" over time - since the plot was not very illuminating for you, this probably isn't happening. I'll admit, you may be able to trick lmer() into doing something more appropriate, though. $\endgroup$
    – Macro
    Commented Aug 17, 2011 at 13:30
  • 2
    $\begingroup$ +1 A good, concise answer addressing all the major points I'd thought to address and more. Regarding packages in R, a Google search of CRAN, for [poisson regression temporal] turns up several packages. The surveillance package may have the functionality desired. This kind of modeling is not uncommon in ecological studies, so it's probably best to find a good package in the ecological nooks of CRAN. $\endgroup$
    – Iterator
    Commented Aug 17, 2011 at 22:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.