1
$\begingroup$

I am new to data analysis with python. I have daily data of flu cases for a five year period which I want to do Time Series Analysis on. Am using the Pandas library. It is easy to plot this data and see the trend over time, however now I want to see seasonality. As it is, the daily data when plotted is too dense (because it's daily) to see seasonality well and I would like to transform/convert the data (pandas DataFrame) into monthly data so I can better see seasonality. Is there an easy way to do this with pandas (or any other python data munging library)?

$\endgroup$
  • 4
    $\begingroup$ Why not smooth the data rather than coarsen them so drastically? $\endgroup$ – whuber Oct 27 '14 at 15:27
  • 1
    $\begingroup$ +1 to @whuber There is no magic to monthly reduction when the data are daily. A month does not have physical or epidemiological meaning. (The fact that many other datasets are reported monthly doesn't mean that you have to mimic that form.) $\endgroup$ – Nick Cox Oct 28 '14 at 9:59
3
$\begingroup$

There are examples of doing what you want in the pandas documentation. In pandas the method is called resample.

monthly_x = x.resample('M')

Or this is an example of a monthly seasonal plot for daily data in statsmodels may be of interest.

import statsmodels.api as sm
import pandas as pd
dta = sm.datasets.elnino.load_pandas().data
dta['YEAR'] = dta.YEAR.astype(int).astype(str)
dta = dta.set_index('YEAR').T.unstack()
dates = map(lambda x : pd.datetools.parse('1 '+' '.join(x)),
                                       dta.index.values)
dta.index = pd.DatetimeIndex(dates, freq='M')
fig = sm.graphics.tsa.month_plot(dta)

enter image description here

$\endgroup$
  • $\begingroup$ What exactly does "resample" do? How does it compute monthly values from daily ones and why would it be reasonable to use it for this particular dataset and for the purpose of detecting seasonality? $\endgroup$ – whuber Oct 28 '14 at 14:05
  • $\begingroup$ I just added the stackoverflow answer to the question as asked. The linked documentation should get a user all the way there. By default, resample takes the mean when downsampling data though arbitrary transformations are possible. $\endgroup$ – jseabold Oct 28 '14 at 14:38
  • $\begingroup$ Incidentally, you could do smoothing using statsmodels and/or pandas but these are software questions. $\endgroup$ – jseabold Oct 28 '14 at 14:40
  • $\begingroup$ As I read it, the heart of this question is "I want to see seasonality." Although this is comprised of two separate follow-on requests--to downsample and to provide Python implementations--the issue that is relevant for this site and (I would argue) of far greater value to the OP concerns how to visualize seasonality in a time series dataset. Don't you think that has to be addressed before recommending a solution? $\endgroup$ – whuber Oct 28 '14 at 14:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for?Browse other questions tagged or ask your own question.