Tell me more ×
Cross Validated is a question and answer site for statisticians, data analysts, data miners and data visualization experts. It's 100% free, no registration required.

I have a vector of values that I would like to report the average in windows along a smaller slide.

For example, for a vector of the following values:

4, 5, 7, 3, 9, 8

A window size of 3 and a slide of 2 would do the following:

(4+5+7)/3 = 5.33
(7+3+9)/3 = 6.33
(9+8)/3 = 5.67

And return a vector of these values:

5.33, 6.33, 5.67

Is there a simple function that will do this for me? If it also returned the indices of the window starts that would be an added bonus. In this example that would be 1,3,5

share|improve this question
4  
Have you seen this? – J. M. Sep 24 '10 at 14:52
Can you give some background on this "slide" idea? – Shane Sep 24 '10 at 15:01
@J.M - I hadn't! Thank you! I'm about to see how it works. – T-Burns Oct 6 '10 at 15:36
@Shane - Yes! I'm sorry that wasn't clear. The slide is the number of positions/indices you move to start computing the next window of averages. So rather than the next window starting after the end of the last there is some overlap when the slide is smaller than your window size. The idea is to smooth out the data points a bit. – T-Burns Oct 6 '10 at 15:39

6 Answers

up vote 11 down vote accepted

Function rollapply in package zoo gets you close:

> require(zoo)
> TS <- zoo(c(4, 5, 7, 3, 9, 8))
> rollapply(TS, width = 3, by = 2, FUN = mean, align = "left")
       1        3 
5.333333 6.333333

It just won't compute the last value for you as it doesn't contain 3 observations. Maybe this will be sufficient for your real problem? Also, note that the returned object has the indices you want as the names of the returned vector.

Your example is making an assumption that there is an unobserved 0 in the last window. It might be more useful or realistic to pad with an NA to represent the missing information and tell mean to handle missing values. In this case we will have (8+9)/2 as our final windowed value.

> TS <- zoo(c(4, 5, 7, 3, 9, 8, NA))
> rollapply(TS, width = 3, by = 2, FUN = mean, na.rm = TRUE, align = "left")
       1        3        5 
5.333333 6.333333 8.500000
share|improve this answer
BTW, I once wrote about a usage of this function to implement the notion of "quantile loess" : r-statistics.com/2010/04/… – Tal Galili Sep 24 '10 at 22:11
You may add a 0 at the end of x (x<-c(x,0)) to get the last element of answer. – mbq Sep 25 '10 at 8:36
1  
@mbq; that is making a strong assumption that the observation is 0. I had been mulling this point and T-Burns is making the same assumption (an unobserved 0). I would prefer perhaps to pad with NA and pass in the na.rm = TRUE argument to mean. The answer won't be the same as what the OP requested, but it seems more useful. I'll edit my answer to include this. – Gavin Simpson Sep 25 '10 at 8:59
@ucfagls Yet this is easy to change and as you said this assumption was made by the OP. On the other hand, I would be even more restrictive and removed the last average. – mbq Sep 25 '10 at 9:24
Thanks! Especially for noting the last value as zero assumption, I hadn't considered that. I definitely care about that last window!! – T-Burns Oct 6 '10 at 15:59
show 1 more comment

This simple line of code does the thing:

((c(x,0,0) + c(0,x,0) + c(0,0,x))/3)[3:(length(x)-1)]

if x is the vector in question.

share|improve this answer
This does not return what the asker wanted, but 5.33 5.00 6.33. However, it looks quite interesting. Can you explain your idea, because I don't get it. – Henrik Sep 24 '10 at 18:14
@Henric I use this trick frequently, yet user1414's code return this roll with slide 1, not 2, as intended by OP. Check out (c(0,0,x)+c(0,x,0)+c(x,0,0))/3 to see what I mean (and how does it work). The proper formula would be: (c(0,0,x)+c(0,x,0)+c(x,0,0))[1:(length(x)-3)*2+1]/3 (we must cut 0-padding at the beginning and select even elements then. – mbq Sep 25 '10 at 8:00

This will get you the window means and the index of the first value of the window:

#The data
x <- c(4, 5, 7, 3, 9, 8)

#Set window size and slide
win.size <- 3
slide <- 2

#Set up the table of results
results <- data.frame(index = numeric(), win.mean = numeric())

#i indexes the first value of the window (the sill?)
i <- 1
#j indexes the row of the results to be added next
j <- 1
while(i < length(x)) {
    #This mean preserves the denominator of 3
    win.mean <- sum(x[i:(i+2)], na.rm = TRUE)/win.size
    #Insert the results
    results[j, ] <- c(i, win.mean)
    #Increment the indices for the next pass
    i <- i + slide
    j <- j + 1
    }

Various caveats apply: haven't tested this against anything but your sample data; I believe that appending to data frames like this can get really slow if you have lots of values (because it'll copy the data.frame each time); etc. But it does produce what you asked for.

share|improve this answer
Please don't downvote without providing a comment. How am I supposed to know what's wrong? – Matt Parker Sep 24 '10 at 19:59
It wasn't me, but this is slow (but not much slower than rollapply). – mbq Sep 25 '10 at 8:06
1  
wasn't me either, but as mentioned by yourself, pre-allocation of the result object will help with the speed issue. One trick, if you don't know, or it is tedious/difficult to determine, the size of the result object you need. Allocate something reasonable, perhaps pre-filling with NA. Then fill in with your loop, but add a check that if you are approaching the limit of the preallocated object, allocate another big chunk, and continue filling. – Gavin Simpson Sep 25 '10 at 9:43
1  
@mbq; Speed of results, whilst important, isn't the only consideration. Instead of having to reinvent the while and handle all the indexes etc in the custom solutions, the one-linear that is rollapply is much easier to understand and grep the intention of. Also, rollapply is likely to have had many more eyeballs checking its code than something I might cook up one afternoon. Horses for courses. – Gavin Simpson Sep 25 '10 at 9:45
ucfagls, thanks for adding some info on pre-allocating when you don't know the ultimate size of the object - that's helpful. And while I agree that rollapply is probably almost definitely the way to go, I a) wanted to provide an example that would calculate the edge cases with less than three values, and b) just wanted to use while(), which I've yet to have a use for ;) – Matt Parker Sep 25 '10 at 14:43

I can do this easily in Matlab and duck while you downvote me:

%given vector x, windowsize, slide 
idx1 = 1:slide:numel(x);
idx2 = min(numel(x) + 1,idx1 + windowsize);  %sic on +1 here and no -1;
cx = [0;cumsum(x(:))];  %pad out a zero, perform a cumulative sum;
rv = (cx(idx2) - cx(idx1)) / windowsize; %tada! the answer!

as a side effect, idx1 is the index of the element in the sum. I am sure this can be easily translated into R. The idiom first:skip:last in Matlab gives the array first, first+skip, first+2skip, ..., first + n skip, where the last element in the array is no greater than last.

edit: I had omitted the averaging part (divide by windowsize).

share|improve this answer
+1 Not tada, rv/windowsize ;-) – mbq Sep 25 '10 at 8:24
This marg... comment box is too narrow for this code, so I've posted a new answer. – mbq Sep 25 '10 at 8:31
Thanks, but MATLAB isn't free!! – T-Burns Oct 6 '10 at 15:33
@T-Burns: octave is free, however; also R is close enough to Matlab that this code can easily be translated. In fact, @mbq did that.. – shabbychef Oct 6 '10 at 17:24

shabbychef's answer in R:

slideMean<-function(x,windowsize=3,slide=2){
 idx1<-seq(1,length(x),by=slide);
 idx1+windowsize->idx2;
 idx2[idx2>(length(x)+1)]<-length(x)+1;
 c(0,cumsum(x))->cx;
 return((cx[idx2]-cx[idx1])/windowsize);
}

EDIT: Indices you're looking for are just idx1... this function can be easily modified to return them also, but it is almost equally fast to recreate them with another call to seq(1,length(x),by=slide).

share|improve this answer
thanks for translating. I figured it would be an easy exercise, and I learned some R from it – shabbychef Sep 26 '10 at 15:59
library(zoo)
x=c(4, 5, 7, 3, 9, 8)
rollmean(x,3)

or

library(TTR)
x=c(4, 5, 7, 3, 9, 8)
SMA(x,3)
share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.