1
$\begingroup$

I am new to data analysis with python. I have daily data of flu cases for a five year period which I want to do Time Series Analysis on. Am using the Pandas library. It is easy to plot this data and see the trend over time, however now I want to see seasonality. As it is, the daily data when plotted is too dense (because it's daily) to see seasonality well and I would like to transform/convert the data (pandas DataFrame) into monthly data so I can better see seasonality. Is there an easy way to do this with pandas (or any other python data munging library)?

$\endgroup$
2
  • 4
    $\begingroup$ Why not smooth the data rather than coarsen them so drastically? $\endgroup$
    – whuber
    Commented Oct 27, 2014 at 15:27
  • 1
    $\begingroup$ +1 to @whuber There is no magic to monthly reduction when the data are daily. A month does not have physical or epidemiological meaning. (The fact that many other datasets are reported monthly doesn't mean that you have to mimic that form.) $\endgroup$
    – Nick Cox
    Commented Oct 28, 2014 at 9:59

1 Answer 1

3
$\begingroup$

There are examples of doing what you want in the pandas documentation. In pandas the method is called resample.

monthly_x = x.resample('M')

Or this is an example of a monthly seasonal plot for daily data in statsmodels may be of interest.

import statsmodels.api as sm
import pandas as pd
dta = sm.datasets.elnino.load_pandas().data
dta['YEAR'] = dta.YEAR.astype(int).astype(str)
dta = dta.set_index('YEAR').T.unstack()
dates = map(lambda x : pd.datetools.parse('1 '+' '.join(x)),
                                       dta.index.values)
dta.index = pd.DatetimeIndex(dates, freq='M')
fig = sm.graphics.tsa.month_plot(dta)

enter image description here

$\endgroup$
4
  • $\begingroup$ What exactly does "resample" do? How does it compute monthly values from daily ones and why would it be reasonable to use it for this particular dataset and for the purpose of detecting seasonality? $\endgroup$
    – whuber
    Commented Oct 28, 2014 at 14:05
  • $\begingroup$ I just added the stackoverflow answer to the question as asked. The linked documentation should get a user all the way there. By default, resample takes the mean when downsampling data though arbitrary transformations are possible. $\endgroup$
    – jseabold
    Commented Oct 28, 2014 at 14:38
  • $\begingroup$ Incidentally, you could do smoothing using statsmodels and/or pandas but these are software questions. $\endgroup$
    – jseabold
    Commented Oct 28, 2014 at 14:40
  • $\begingroup$ As I read it, the heart of this question is "I want to see seasonality." Although this is comprised of two separate follow-on requests--to downsample and to provide Python implementations--the issue that is relevant for this site and (I would argue) of far greater value to the OP concerns how to visualize seasonality in a time series dataset. Don't you think that has to be addressed before recommending a solution? $\endgroup$
    – whuber
    Commented Oct 28, 2014 at 14:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.