Tell me more ×
Cross Validated is a question and answer site for statisticians, data analysts, data miners and data visualization experts. It's 100% free, no registration required.
up vote 4 down vote favorite
2

I have some real data from the past. The data show application demand (for example cpu demand) at a certain time slot. The data looks like this for example:

3,2,1,5,7,8,9,1,3,12,4,5

These 12 values show the application demand of cpu in first 12 hours of a day respectively: 3 was the demand between 00:00 and 01:00m, 2 the demand between 01:00 - 02:00 etc...

So what I have is a bunch of values and what I would like to do with this is a forecasting or estimation of application demand in the future. Say I have 120 values showing the demand of 10 days of 12 hours each. Based on these data, I want to estimate with which probability the demand values will be more or less than the previous ones.

How can I achieve this with R? I think this question is not directly related to R, but thought there are people with great knowledge in general who can give me some concrete ideas.

Thanks for your precious time!

share|improve this question
1  
This is a very, very broad question. Try to narrow it down a little as to what your specific question is. Perhaps try to explain what you have already tried and where you got stuck... – Andrie Aug 23 '11 at 13:32
@Andrie I would like to do linear programming when I have my estimation => stochastic programming with recourse costs => say my estimation was below the real value, I have a shortage, and overage when vice versa etc... These shortages and overages are connected with additional costs of course. – Bob Aug 23 '11 at 13:41

migrated from stackoverflow.com Aug 23 '11 at 15:36

4 Answers

up vote 4 down vote

Start with the forecast package in R. Both auto.arima and ets have predict and forecast methods that are useful.

Here is an example:

#Make a periodic dataset
set.seed(1)
data <- c(3,2,1,5,7,8,9,1,3,12,4,5)
data <- rep(data,5)
data <- data+seq(1,length(data)) #Add trend
data <- data+rnorm(length(data))*5 #Add noise
data <- ts(data,frequency=12)
plot(data)

#Build models
library(forecast)
model1 <- StructTS(data)
model2 <- ets(data)
model3 <- auto.arima(data)
model4 <- stl(data,s.window='periodic')

#Test accuracy (on training data)
round(accuracy(model1),4)
round(accuracy(model2),4)
round(accuracy(model3),4)
p4 <- model4$time.series
p4 <- p4[,'seasonal']+p4[,'trend']+p4[,'remainder']
round(accuracy(data,p4),4)  #This model needs to be tested OUT of sample

#Forecast 1 period
f1 <- forecast(model1,h=12) #this fails for some reason
f2 <- forecast(model2,h=12)
f3 <- forecast(model3,h=12)
f4 <- forecast(model4,h=12)

par(mfrow = c(2,2))
plot(f1)
plot(f2)
plot(f3)
plot(f4)
share|improve this answer
up vote 2 down vote

What you are doing is time series analysis, and there are many packages for that. Take a look at the Task View to get you started.

Probably the simplest analysis to do (and you should always start simple) is to decompose the series into a daily effect, a trend and irregular components. Take a look at the stl function.

share|improve this answer
I already decomposed the data and plotted it already couple of times by aggregating the values. I can see a clear indication that there is a pattern in it, but don't know how to estimate the future using the existing data. I have searched a lot and found out that there are many methods for doing so and none of them is perfect, but what I want is I just want to try as many as possible with R without too much effort and gonna chooes one of them as suitable one for my case – Bob Aug 23 '11 at 13:43
The standard way of predicting things with a model is to use a predict method. Take a look at methods(predict) and predict.StructTS. Which method to use is a broad and tricky question. Like Andrie says, tell us what you've tried already. – Richie Cotton Aug 23 '11 at 14:08
I already tried with mean, but mean is somehow not good representative. I also tried with forecast function with different prediction algorithms. I am trying to create a linear programming model which works with stochastic variable. (the stochastic variable is the application demand vector) I have an infinite number resources which can be assigned anytime to satisfy the demand, but must be paid for the whole period, in our case 1 hour. The goal is to minimize the cost in case of uncertain demand in the future doing forecast based on the data from the past. – Bob Aug 23 '11 at 14:47
1  
This makes no sense. "Linear programming" is an optimization technique, not a structural model. A linear regression is a (regression) method to satisfy a linear model, albeit it is often used synonymously with nouns, verbs, and various verb phrases "run an OLS", "do an OLS". If you are doing linear programming, then your problem has more complexity than you've let on so far. – Iterator Aug 23 '11 at 15:18
2  
@Iterator I think we can make a lot of sense of Bob's objective if we allow that he isn't speaking statistical jargon. Based on his time series data, he wants to know the optimal way to deploy resources for a given future period. It is debatable whether that optimum is found with linear programming, but that an optimum ought to exist, and ought to be determinable from the data (along with appropriate assumptions and models) is intuitively clear. This makes it a stochastic control problem. – whuber♦ Aug 26 '11 at 22:16
show 2 more comments
up vote 0 down vote

If you have reliable historical data, you can use an MCMC approach

check the following articles out,

http://lpenz.github.com/articles/df0pred-2/index.html

http://lpenz.github.com/articles/df0pred-3/index.html

He does prediction of hard drive space requirements using MCMC.

share|improve this answer
and also the first post to this series lpenz.github.com/articles/df0pred-1/index.html – ECII Aug 27 '11 at 15:06
up vote 0 down vote

"I have 120 values showing the demand of 10 days of 12 hours each." . This is a mixed frequency problem where you might have an integrated model containing an ARIMA component describing the within day structure and a Transfer Function component ( generalization of regression ) to deal with the daily effects. These "daily effects" might be impacted by heretofore unknown ( but statistically identifiable via Intervention Detection ) Pulses, Level Shifts and Local Time Trends. Furthermore there may be day-of-the-week; day-of-the-month ; week/month of-the-year effects. There might also be patterns before ,on and after known events. Additionally there may be evidented non-constant error variance suggesting GLS/GARCH or even changes in parameters over time.

After forming such a model with proven Gaussian errors it seems to me a straightforward application using the confidence limits of the forecast vis-a-vis the last value to assess the probability that you are seeking.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.