2
$\begingroup$

I am a little new to time series in general, however, I have 5 years of weekly sales data and another variable of interest.

I am trying to see if there is a trend in the sales data and the second variable. The sales data has some seasonality trends I would like to correct for. What I have done thus far is normalise the data for both variables using (value-min)/(max-min) and simply just plotted these two just as the plot below.

ggplot(df, aes(x = WEEKref)) +
  geom_line(aes(y = Adjmm, colour = "red")) +
  geom_line(aes(y = dollarmm, colour = "blue"))

I have looked into the forecast package in R but at this stage I am not trying to forecast anything, I just want to see if there is an underlying relationship between the two variables.

I have been following this tutorial: https://anomaly.io/seasonally-adjustement-in-r/

Do you have any advice on where I should be looking?

enter image description here

Data (R dput):

df <- structure(list(WEEKref = structure(c(13885, 13892, 13899, 13906, 
13913, 13920, 13927, 13934, 13941, 13948, 13955, 13969, 13976, 
13983, 13990, 13997, 14004, 14011, 14018, 14025, 14032, 14039, 
14046, 14053, 14060, 14067, 14074, 14081, 14088, 14095, 14102, 
14109, 14116, 14123, 14130, 14137, 14144, 14151, 14158, 14165, 
14172, 14179, 14186, 14193, 14200, 14207, 14214, 14221, 14228, 
14235, 14242, 14249, 14256, 14263, 14270, 14277, 14284, 14291, 
14298, 14305, 14312, 14319, 14326, 14333, 14340, 14354, 14361, 
14368, 14375, 14382, 14389, 14396, 14403, 14410, 14417, 14424, 
14431, 14438, 14445, 14452, 14459, 14466, 14473, 14480, 14487, 
14494, 14501, 14508, 14515, 14522, 14529, 14536, 14543, 14550, 
14557, 14564, 14571, 14578, 14585, 14592, 14599, 14606, 14613, 
14620, 14627, 14634, 14641, 14648, 14655, 14662, 14669, 14676, 
14683, 14690, 14697, 14711, 14718, 14725, 14732, 14739, 14746, 
14753, 14760, 14767, 14774, 14781, 14788, 14795, 14802, 14809, 
14816, 14823, 14830, 14837, 14844, 14851, 14858, 14865, 14872, 
14879, 14886, 14893, 14900, 14907, 14914, 14921, 14928, 14935, 
14942, 14949, 14956, 14963, 14970, 14977, 14984, 14991, 14998, 
15005, 15012, 15019, 15026, 15033, 15040, 15047, 15054, 15061, 
15068, 15075, 15082, 15096, 15103, 15110, 15117, 15124, 15131, 
15138, 15145, 15152, 15159, 15166, 15173, 15180, 15187, 15194, 
15201, 15208, 15215, 15222, 15229, 15236, 15243, 15250, 15257, 
15264, 15271, 15278, 15285, 15292, 15299, 15306, 15313, 15320, 
15327, 15341, 15348, 15355, 15362, 15369, 15376, 15383, 15390, 
15397, 15404, 15411, 15418, 15425, 15432, 15446, 15453, 15460, 
15467, 15474, 15481, 15488, 15495, 15502, 15509, 15516, 15523, 
15530, 15537, 15544, 15551, 15558, 15565, 15572, 15579, 15586, 
15593, 15600, 15607, 15614, 15621, 15628, 15635, 15642, 15649, 
15656, 15663, 15670, 15677, 15684, 15691, 15698), class = "Date"), 
    Adjmm = c(0.3788241497815, 0.344088491487985, 0.284635983406239, 
    0.312477827282932, 0.34381374862743, 0.336002992372229, 0.354656867077484, 
    0.349050664458906, 0.392424643476541, 0.369267137912271, 
    0.364488601466493, 0.34886704827454, 0.365499883823658, 0.386083711690411, 
    0.38498138402016, 0.366511206862376, 0.361317149045795, 0.336964338124474, 
    0.35636750599285, 0.31083166862728, 0.300932199454407, 0.304396701434002, 
    0.310237921377734, 0.312117409074138, 0.273016559242622, 
    0.249753179939609, 0.262325406729579, 0.246190452353024, 
    0.270541737716149, 0.266483468146233, 0.320235636498374, 
    0.307169210216285, 0.293111679626758, 0.306920605252806, 
    0.3237981606772, 0.304198724661826, 0.288063810966823, 0.243022051048717, 
    0.220353842849565, 0.163929038924983, 0.143387479190623, 
    0.102999404035606, 0.163433374896997, 0.154772211481501, 
    0.122549757758648, 0, 0.0307910122979721, 0.0512170567657967, 
    0.0619050767993455, 0.0803518213832529, 0.0970567446422854, 
    0.109011608541848, 0.137512961396686, 0.122153865236623, 
    0.151605295053837, 0.138462997677511, 0.170368745519054, 
    0.157305634783443, 0.178523362591438, 0.158097338464389, 
    0.144717501506692, 0.186440419741675, 0.16126417352895, 0.154138779378106, 
    0.182798586877478, 0.192140698448953, 0.193565793551742, 
    0.217059369416012, 0.237818534771326, 0.235013236658236, 
    0.231246124955054, 0.232608712853395, 0.231887286553305, 
    0.2181815293428, 0.22339135126073, 0.252886757835516, 0.264829152839431, 
    0.268355877252611, 0.277493116528868, 0.283343998341169, 
    0.276771710569553, 0.291118710741412, 0.279897518289053, 
    0.309152537573837, 0.319411774050674, 0.306587748795404, 
    0.340010430343084, 0.355800181411244, 0.335121402802426, 
    0.331514535732061, 0.354197124859799, 0.381688802278769, 
    0.370948648836498, 0.357242911966769, 0.365578521263423, 
    0.388581782723778, 0.382810921524005, 0.369505897940197, 
    0.403810392806216, 0.414550485226159, 0.403970617098257, 
    0.415833072852747, 0.430981336870242, 0.402527927224284, 
    0.405573714331254, 0.38144845567032, 0.419359593858168, 0.393951889956077, 
    0.413588854703051, 0.423046583244941, 0.441080369395814, 
    0.411344453486371, 0.42280599254718, 0.448293920469559, 0.442523120292115, 
    0.454786248650974, 0.422565686620283, 0.432584495125567, 
    0.432609331212984, 0.466306959028979, 0.462823621490868, 
    0.463390681642387, 0.47311125369091, 0.476108426343396, 0.500895533171707, 
    0.511182778897006, 0.496521188284535, 0.463633733574035, 
    0.512641070146122, 0.498870263134176, 0.522604287310775, 
    0.505917569046205, 0.478214225512853, 0.512155108668256, 
    0.497655206883693, 0.497088309458382, 0.531190901782468, 
    0.532729905230494, 0.533782926859879, 0.534107036783629, 
    0.516934275678089, 0.554600836376093, 0.559218070468706, 
    0.577524951877284, 0.569262589714772, 0.543179409325428, 
    0.537752164853527, 0.557597968347027, 0.523738488976916, 
    0.534755032882593, 0.537752164853527, 0.539777292505626, 
    0.54236939894614, 0.534107036783629, 0.542855299401678, 0.530866791858718, 
    0.506646643478046, 0.491418053736384, 0.509805789729303, 
    0.495954046769912, 0.481859577331287, 0.491418053736384, 
    0.501057323703495, 0.490688979304542, 0.480077359225405, 
    0.489311847751408, 0.514665974049686, 0.50526957294724, 0.521956372574914, 
    0.547040861547397, 0.515437560703232, 0.532023022585198, 
    0.523524015835457, 0.517087746832519, 0.503142741415465, 
    0.49588134883672, 0.494066036288392, 0.48630963058649, 0.509248981328856, 
    0.492663295698355, 0.483916517956949, 0.518573050632076, 
    0.496789035622048, 0.43737781931545, 0.466258344574494, 0.463782750098536, 
    0.456438916354926, 0.433252160754861, 0.434077243649116, 
    0.459657010174843, 0.47599506719911, 0.502647606247653, 0.483173855886782, 
    0.490352766619111, 0.500667248643387, 0.500337239894616, 
    0.494561008729997, 0.53524123844977, 0.532353194060176, 0.559583289100621, 
    0.577654217508351, 0.56428656467544, 0.581862520641565, 0.634094927919714, 
    0.620479687162121, 0.626915956165058, 0.618912064242354, 
    0.605709619191587, 0.642841542934913, 0.647462357004091, 
    0.657694194455743, 0.651340732751637, 0.67749817731392, 0.724697076519945, 
    0.740705002750785, 0.740209948946076, 0.745655886591061, 
    0.744335770232873, 0.735424028798633, 0.754954184131172, 
    0.80537990334999, 0.77215637833949, 0.736065576871626, 0.744835319349063, 
    0.726621232560302, 0.751918404685962, 0.782949763592383, 
    0.793743230208379, 0.895606723952665, 0.891053197170281, 
    0.899823021010821, 0.900160230394274, 0.954464874561588, 
    0.949742743087477, 0.939961169051924, 0.95244129280846, 0.949742743087477, 
    0.995615159272445, 0.952778522532688, 0.933721147855208, 
    0.966776312263293, 0.963234698402128, 1, 0.968294201985898, 
    0.948056309695473, 0.949742743087477, 0.921072500770051, 
    0.936419636553863, 0.955476767142031, 0.96492119281646, 0.984990012577312, 
    0.984484401909893, 0.952103839335696, 0.951429318864912), 
    dollarmm = c(0.149697356812732, 0.0244998111324886, 0.0550537737139359, 
    0.0274508134712055, 0.213984247872558, 0.0783422018092716, 
    0.0310253280164216, 0.0541816334844162, 0.0806492605345567, 
    0.0877855302949306, 0.152766825529036, 0.168423392180753, 
    0.172908899350928, 0.185482743591095, 0.213353668598008, 
    0.229097493059307, 0.261158363124192, 0.282936314890266, 
    0.340039521860067, 0.592897079173477, 0.437728811188524, 
    0.413066850628898, 0.490885152462683, 0.456852949424961, 
    0.492964518353547, 0.972882256833953, 0.420276672512582, 
    0.439876590158875, 0.443357095278529, 0.480802216652516, 
    0.426624787626307, 0.398178487457366, 0.395484743345358, 
    0.61293865090702, 0.42777529076777, 0.283409977946298, 0.295947790026711, 
    0.317550712975473, 0.299640010568774, 0.300719638221296, 
    0.291610161608615, 0.247095733415861, 0.336036187889497, 
    0.228521023231508, 0.213955172137217, 0.273338090020996, 
    0.485758317106326, 0.143782632825279, 0.169335703112876, 
    0.328666970899003, 0.491065758190125, 0.506609550726262, 
    0.112120844942705, 0.17295508953821, 0.16192922040378, 0.388163795230051, 
    0.227206496783438, 0.145802804370329, 0.171606159981382, 
    0.154086848847159, 0.210374588670803, 0.197064573460434, 
    0.288013610146669, 0.220217969236187, 0.273558409751774, 
    0.288506341574192, 0.326273737259306, 0.317036580243605, 
    0.376241375996475, 0.381665974567176, 0.707453475415865, 
    0.485169961326965, 0.420451063363391, 0.449702773878119, 
    0.502821656395841, 0.491841956261615, 1, 0.44763479957363, 
    0.458798662510525, 0.48161107578401, 0.496614239968388, 0.47619764751241, 
    0.460418831412368, 0.447895106385658, 0.392285202440178, 
    0.655937896884758, 0.430264772601089, 0.323869738229137, 
    0.303764772399812, 0.258602815268962, 0.260930408982457, 
    0.240764266638331, 0.250696875411684, 0.337172813493932, 
    0.227452403443811, 0.229145352317625, 0.28521986301974, 0.478231444703654, 
    0.132760650342865, 0.127452787871597, 0.296331593970631, 
    0.47654036029283, 0.483701772054131, 0.0900944558373256, 
    0.163929433720004, 0.142555512496348, 0.164139722545731, 
    0.35237013136599, 0.235904612124217, 0.115107342701831, 0.128553880389704, 
    0.168914894757668, 0.157147609673816, 0.218033903861127, 
    0.192778260148724, 0.264010141661686, 0.265934476984103, 
    0.209431740094857, 0.244681209115455, 0.288711990429523, 
    0.289509990005749, 0.315092875222347, 0.686681695697921, 
    0.452317417206126, 0.374537214449401, 0.473850571102317, 
    0.451832204837682, 0.836981605987354, 0.51920428542675, 0.446969106246101, 
    0.421502554574427, 0.41709701215027, 0.390323036410375, 0.365705653465875, 
    0.333904920716383, 0.300121453991199, 0.579486246422688, 
    0.321055545570556, 0.215519275318642, 0.215565915142833, 
    0.208650826721134, 0.214130384575884, 0.232400949526744, 
    0.17072512824086, 0.241964382857466, 0.156667771761189, 0.145988572682793, 
    0.199009246024986, 0.371723057102618, 0.0511910889931185, 
    0.0947298145120868, 0.154669507656273, 0.357442986752192, 
    0.340553948856928, 0, 0.0576641678554561, 0.0625454708565096, 
    0.0770529539010844, 0.258504104665886, 0.099505662356445, 
    0.0741165588765761, 0.0619227096142703, 0.0892981276862553, 
    0.0794331221751312, 0.144165085504355, 0.0616202640553713, 
    0.131826476698887, 0.137209722798776, 0.166437178815794, 
    0.166687664239444, 0.218930682309907, 0.222712288785976, 
    0.238746934161925, 0.513927986995597, 0.392680063685442, 
    0.314372316775567, 0.37271347773425, 0.330054166675406, 0.735454399633936, 
    0.484249008420553, 0.329267937698866, 0.35492302612805, 0.355395255525041, 
    0.346946500338582, 0.314414354295281, 0.315905256117267, 
    0.318457847197856, 0.512463359506227, 0.311027552565597, 
    0.192393768801782, 0.177813119709567, 0.192887375961921, 
    0.20094859121612, 0.190638711181672, 0.142654512656176, 0.188030459289294, 
    0.131392002677444, 0.1354966579701, 0.178427989941778, 0.358665186970456, 
    0.0421970278185858, 0.0660661513870076, 0.12819972894051, 
    0.333626779091713, 0.0415224265012151, 0.0906564289945531, 
    0.0713307780915137, 0.0938540183343052, 0.321155525044278, 
    0.184479613065924, 0.108561719279517, 0.102441840801919, 
    0.107796585571731, 0.130178451495829, 0.265694549788369, 
    0.208895233807108, 0.214943212753592, 0.213012834405297, 
    0.240648159088921, 0.240465787771525, 0.294307615813644, 
    0.302326040229749, 0.360467454111782, 0.657805801598166, 
    0.467494633248041, 0.352543088557757, 0.468025614416299, 
    0.452103599554613, 0.625060089794185, 0.828415542650317, 
    0.402265340895058, 0.389391661815002, 0.427983230181801, 
    0.444962482661289, 0.404995279601097, 0.38580728350725, 0.382196506098525, 
    0.601431192394729, 0.419878449584049, 0.326484958316528, 
    0.288920671391334, 0.273866371013686, 0.262324215127644, 
    0.194923630854379, 0.221234530015598, 0.247565234387846, 
    0.229392396017928, 0.164468590746803, 0.218632123403024, 
    0.437144925882923, 0.0830346125807046, 0.136073219608577, 
    0.134629124753297, 0.388556693139287)), row.names = c(NA, 
-254L), class = c("tbl_df", "tbl", "data.frame"))
$\endgroup$
2
$\begingroup$

Your reference https://anomaly.io/seasonally-adjustement-in-r/ is inadequate to deal with causals , model identification and a ton of other issues e.g. detection of level shifts , seasonal pulses , pulses et al .

Standard ARIMA/SARIMA/TRANSFER FUNCTION/DYNAMIC REGRESSION model identification using the acf/pacf .. ccf requires that the are no deterministic structure in the data such as weekly effects. Your data has 26 significant weekly effects (see enter image description here and enter image description here and some unusual observations (pulses) and a small ARIMA structure while supporting the significance of your predictor series.

Ordinary multiple regression often fails when there are latent factors waiting to be discovered. See https://autobox.com/cms/index.php/afs-university/intro-to-forecasting/doc_download/18-regression-vs-box-jenkins for a discussion of reression vs box-jenkins when dealing with autocorrelated data.

The residuals from the model suggest reasonable sufficiency enter image description here and the forecast plot visually suggest that a certain # of weeks in the year have little autocorrelation ( flat forecast) while some 26 weeks are visually informative.

The Actual/Fit and Forecast graph is here

enter image description here

Combining fixed effects (26 weekly indicators ) , your predictor series , and memort while adjusting for pulses/anomalies leads to a reasonable resolution.

By the way I used AUTOBOX ( a program that I have helped to develop which is available in R )in a totally autonatic fashion to form this result in 2 seconds !).

If there are no level shifts and no seasonal pulses/indicators and no deterministic trends and no variance changes in the model error process and no changes in model parameters over time then simple ARIMA/ARMAX models might be sufficient. In this case notsomuch .

Finally the Actual/Cleansed graph provides an insight into the data and the strong need to research why the identified anomalies actually happened. This kind of exploratory data leads to the identification of additional input series which may be needed.

  • enter image description here

You had mused about trend. There is no trend BUT there is an upwards level shift at period 209. Sometimes people misuse the word trend when they mean (joke !) a shift in the level of the series.

Even though you have no interest in forecasting , it is often educational/informative to look at the forecast graph as it tells about the model in a visual way.

enter image description here

enter image description here

$\endgroup$
  • $\begingroup$ Thanks for your detailed response! Do you have the R code at hand for the AUTOBOX implementation I would love to go through it line by line to understand it more in depth $\endgroup$ – user113156 Nov 9 '18 at 21:34
  • 1
    $\begingroup$ The R version of AUTOBOX is an executable version . If you wish to talk offline about this please software please contact me as software discussions are to be avoided on SE. In this way I can give you a step-by-step discussion and perhaps even share the analysis details. $\endgroup$ – IrishStat Nov 9 '18 at 21:40
  • 1
    $\begingroup$ If you are happy with my answer , please accept it and close the question $\endgroup$ – IrishStat Nov 28 '18 at 22:14
  • $\begingroup$ Done! I will contact you when I am ready to push this model forward! Thanks again for your time! $\endgroup$ – user113156 Nov 28 '18 at 22:17
  • $\begingroup$ glad to be of help .... $\endgroup$ – IrishStat Nov 28 '18 at 22:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.