13
$\begingroup$

I am trying to grasp the dynamic time warping measure for comparing time series together. I have three time series datasets like this:

T1 <- structure(c(0.000213652387565, 0.000535045478866, 0, 0, 0.000219346347883, 
0.000359669104424, 0.000269469145783, 0.00016051364366, 0.000181950509461, 
0.000385579332948, 0.00078170803205, 0.000747244535774, 0, 0.000622858922454, 
0.000689084895259, 0.000487983408564, 0.000224744353298, 0.000416449765747, 
0.000308388157895, 0.000198906016907, 0.000179549331179, 9.06289650172e-05, 
0.000253506844685, 0.000582896161212, 0.000386473429952, 0.000179839942451, 
0, 0.000275608635737, 0.000622665006227, 0.00036075036075, 0.00029057097196, 
0.000353232073472, 0.000394710874285, 0.000207555002076, 0.000402738622634, 
0, 0.000309693403531, 0.000506521463847, 0.000226988991034, 0.000414164423276, 
9.6590360282e-05, 0.000476689865573, 0.000377572210685, 0.000378967314069, 
9.25240562546e-05, 0.000172309813044, 0.000447627573859, 0, 0.000589333071408, 
0.000191699415317, 0.000362943471554, 0.000287549122975, 0.000311688311688, 
0.000724112961622, 0.000434656621269, 0.00122292103424, 0.00177549812586, 
0.00308008213552, 0.00164338537387, 0.00176056338028, 0.00180072028812, 
0.00258939580764, 0.00217548948513, 0.00493015612161, 0.00336344416683, 
0.00422716412424, 0.00313360554553, 0.00540144648906, 0.00425728829246, 
0.0046828437633, 0.00397219463754, 0.00501656412683, 0.00492700729927, 
0.00224424911165, 0.000634696755994, 0.00120550276557, 0.00125313283208, 
0.00164551010813, 0.00143575017947, 0.00237006940918, 0.00236686390533, 
0.00420336269015, 0.00329840900272, 0.00242005185825, 0.00326554846371, 
0.006217237596, 0.0037103784586, 0.0038714672861, 0.00455830066551, 
0.00361747518783, 0.00304147465438, 0.00476801760499, 0.00569875504121, 
0.00583855136233, 0.0050566695728, 0.0042220072126, 0.00408237321963, 
0.00255222610833, 0.00123507616303, 0.00178136133508, 0.00147434637311, 
0.00126742712294, 0.00186590371937, 0.00177226406735, 0.00249154653853, 
0.00549127279859, 0.00349072202829, 0.00348027842227, 0.00229555236729, 
0.00336862367661, 0.00383477593952, 0.00273999412858, 0.00349618180145, 
0.00376108175875, 0.00383351588171, 0.00368928059028, 0.00480028982882, 
0.00388823582602, 0.00745054380406, 0.0103754506287, 0.00822677278011, 
0.00778350981989, 0.0041831792162, 0.00537228238059, 0.00723645609231, 
0.0144428396845, 0.00893333333333, 0.0106231171714, 0.0158367059652, 
0.01811729548, 0.0207095263821, 0.0211700064641, 0.017604180993, 
0.0165804327375, 0.0188679245283, 0.0191859923629, 0.0269251008595, 
0.0351239669421, 0.0283510318573, 0.0346557651212, 0.0270022042616, 
0.0260845175767, 0.0349758630112, 0.0207069247809, 0.0106362024818, 
0.00981093510475, 0.00916507201128, 0.00887198986058, 0.0073929115025, 
0.00659077291791, 0.00716191546131, 0.00942304513143, 0.0106886280007, 
0.0123527175979, 0.0171022290546, 0.0142909490656, 0.0157642220699, 
0.0265140538974, 0.0194395354708, 0.0241685144124, 0.0229897123662, 
0.017921889568, 0.0155115839714, 0.0145263157895, 0.017609281127, 
0.0157671315949, 0.0190258751903, 0.0138453217956, 0.00958058335108, 
0.0122924304507, 0.00929741151611, 0.00885235535884, 0.00509319462505, 
0.0061314863177, 0.0063104189044, 0.00729117134253, 0.010843373494, 
0.0217755443886, 0.0181687353841, 0.0155402963498, 0.017310022503, 
0.0214746959003, 0.026357827476, 0.0194751217195, 0.0196820590462, 
0.0184317400812, 0.0130208333333, 0.0128666035951, 0.0120045731707, 
0.0122374253228, 0.00874940561103, 0.0114368092263, 0.00922893718369, 
0.00479041916168, 0.00644107774653, 0.00775830595108, 0.00829578041786, 
0.00681348095875, 0.00573782551125, 0.00772002058672, 0.0112488083889, 
0.00908907291456, 0.0157722638969, 0.00994270306707, 0.0134179772039, 
0.0126050420168, 0.0113648781554, 0.0153894803415, 0.0126959699913, 
0.0116655865198, 0.0112065745237, 0.0122006737686, 0.010251878038, 
0.010891174691, 0.0148273273273, 0.0138516532618, 0.0136552722011, 
0.00986993819758, 0.0097852677358, 0.00889011089726, 0.00816723383568, 
0.00917641660931, 0.00884466556108, 0.0182179529646, 0.0183156760639, 
0.0217806648835, 0.0171099125907, 0.0186579938377, 0.019360390076, 
0.0144603654529, 0.0177730696798, 0.0153226598566, 0.0134016909516, 
0.0126480805202, 0.0115501519757, 0.0127156322248, 0.0124326204138, 
0.0240245215806, 0.0130234933606, 0.0144222706691, 0.00854005693371, 
0.0053560967445, 0.00504132231405, 0.00288778877888, 0.00593526847816, 
0.00455653279644, 0.00433014040152, 0.00535770564135, 0.0131095962244, 
0.0126319758673, 0.0154982879798, 0.0125940464508, 0.0169948745616, 
0.0257535512184, 0.0256175663312, 0.0265191262043, 0.0228974403622, 
0.0193122555411, 0.0165794768612, 0.015658837248, 0.0168208578638, 
0.0129912843282, 0.0119498443154, 0.0112663755459, 0.00838112042347, 
0.00925767186696, 0.0113408269771, 0.0210861519924, 0.0156036134684, 
0.0121687119728, 0.011006497812, 0.0107891491985, 0.0134615384615, 
0.0147229755909, 0.015756893641, 0.0176257128046, 0.016776075857, 
0.0169553999263, 0.0179193118984, 0.0190055672874, 0.0183088625509, 
0.0155489923558, 0.0152507401094, 0.0160748342567, 0.0161532350605, 
0.0139190952588, 0.0161469457497, 0.0118186629035, 0.0109259765092, 
0.00950587391265, 0.00928986154533, 0.00815520645549, 0.00702576112412, 
0.00709539362541, 0.00827287768869, 0.0104688211197, 0.0130375888927, 
0.0160891089109, 0.0188415910677, 0.0203265044814, 0.0183175033921, 
0.0139940353292, 0.0124648170487, 0.0131685758095, 0.00957428620277, 
0.0119647893342, 0.00835800104475, 0.0101892285298, 0.00904207699194, 
0.00772134522992, 0.00740740740741, 0.00776823249863, 0.00642254601227, 
0.00484237572883, 0.00361539964823, 0.00414811817078, 0.00358072916667, 
0.00433306007729, 0.00485008818342, 0.00905280804694, 0.00931847250137, 
0.00779271381259, 0.00779912497622, 0.00908230842006, 0.0058152538582, 
0.0102777777778, 0.00807537012113, 0.00648535564854, 0.0145492582731, 
0.00694127317563, 0.00759878419453, 0.00789242911429, 0.00635050701629, 
0.00785233530492, 0.00607964332759, 0.00531968282646, 0.00361944157187, 
0.00305157155935, 0.00276327909119, 0.00318820364651, 0.00184464029514, 
0.00412550211703, 0.00516567972786, 0.00463655399342, 0.00702897308418, 
0.0100714154917, 0.00791168353266, 0.00959190791768, 0.00736, 
0.00738007380074, 0.012573964497, 0.0117919562013, 0.00842919476398, 
0.00778887565289, 0.00623967700496, 0.0062232955601, 0.00447815755803, 
0.00511135450894, 0.00502557659517, 0.00330328263712), .Tsp = c(1, 
15.9583333333333, 24), class = "ts")

T2 <- structure(c(0, 0, 0, 0, 0.000109673173942, 0, 0, 0, 0, 0, 0, 
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 9.66183574879e-05, 0, 
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 9.43930526713e-05, 
0, 0, 0, 8.95255147717e-05, 0, 0, 0, 0, 0.000191699415317, 0.000207792207792, 
0, 0, 0, 0.00019727756954, 0.000205338809035, 0.000205423171734, 
0.000704225352113, 0.000450180072029, 0.000493218249075, 0.000120860526952, 
0.000410846343468, 0.000384393619066, 0.000643264105863, 0.000189915487608, 
0.000915499404925, 0.000185099490976, 0.000936568752661, 0.000451385754266, 
0.000757217226692, 0.000273722627737, 0.000187020759304, 0.000211565585331, 
0.000141823854772, 9.63948332369e-05, 0.000117536436295, 0.000287150035894, 
0, 0, 0.000400320256205, 0.000388048117967, 0.000345721694036, 
0.000296868042155, 0.000609533097647, 0.000424043252412, 0.000290360046458, 
0.000546996079861, 0.000556534644282, 0.00036866359447, 0.000275077938749, 
0.000964404699281, 0.00152310035539, 0.00113339145597, 0.00061570938517, 
0.000362877619523, 0.000472634464505, 0.000102923013586, 0.000187511719482, 
0.000294869274622, 0.00011522064754, 0.000248787162582, 0, 0.00035593521979, 
0.000392233771328, 0.000551166636046, 0.000165727543918, 0.000143472022956, 
0.00012030798845, 0.000438260107374, 0.000195713866327, 0.000184009568498, 
0.000537297394108, 0.000365096750639, 0.000102480016397, 0.000452857531021, 
0.000180848177955, 0.000770745910765, 0.00219818869252, 0.000357685773048, 
0.000362023712553, 0.000660501981506, 0.000419709560984, 0.000488949735967, 
0.00177758026886, 4e-04, 0.000475661962898, 0.000879816998064, 
0.0014942099365, 0.00378173960022, 0.00274725274725, 0.00192545729611, 
0.0016462841016, 0.00176238855484, 0.00260780478718, 0.00447289949132, 
0.0034435261708, 0.00290522941294, 0.002694416055, 0.0041329904482, 
0.00729244577412, 0.0296930503689, 0.00982375036117, 0.00453023439039, 
0.00327031170158, 0.00221573169503, 0.00211237853823, 0.00108719286801, 
0.00131815458358, 0.000983008004494, 0.00132253265002, 0.00227790432802, 
0.00247054351957, 0.00307455803228, 0.0029314767314, 0.00222755311857, 
0.00492610837438, 0.00454430699318, 0.00753880266075, 0.00671845475541, 
0.00590490003108, 0.00288356368698, 0.00294736842105, 0.00248601615911, 
0.00197089144936, 0.00326157860404, 0.00302866414278, 0.00202256759634, 
0.00258788009489, 0.00169043845747, 0.00137000737696, 0.000433463372345, 
0.000908368343363, 0.000805585392052, 0.00142653352354, 0.00189328743546, 
0.00558347292016, 0.00161899622234, 0.00162631008312, 0.00276960360048, 
0.00585673524553, 0.00519169329073, 0.0045125282033, 0.00562344544176, 
0.00322815786733, 0.00330528846154, 0.00255439924314, 0.00285823170732, 
0.00240894199268, 0.00218735140276, 0.00201826045171, 0.00168701002282, 
0.000460617227084, 0.00127007166833, 0.00109529025192, 0.000819336337567, 
0.00158170093685, 0.000588494924231, 0.00120089209127, 0.00305052430887, 
0.00161583518481, 0.00211579149837, 0.0010111223458, 0.00346270379455, 
0.00228091236495, 0.00207627581685, 0.00295140718878, 0.0022121765894, 
0.00240718451995, 0.00224131490474, 0.0031867431485, 0.00176756517897, 
0.00233382314807, 0.00178303303303, 0.00169794459339, 0.00162778079219, 
0.000737939304492, 0.00135906496331, 0.000733205022454, 0.000875060768109, 
0.00114705207616, 0.000967385295744, 0.00182179529646, 0.00359130903214, 
0.00420328620558, 0.00446345545843, 0.00376583361862, 0.00659687365553, 
0.00433810963586, 0.00353107344633, 0.00333955407131, 0.00341788091383, 
0.0024939877082, 0.00538428137212, 0.00906989151698, 0.00773778473309, 
0.0210421671775, 0.00859720803541, 0.00511487506289, 0.00406669377796, 
0.00117164616286, 0.00206611570248, 0.00107260726073, 0.00148381711954, 
0.000741761152909, 0.00104973100643, 0.00110305704381, 0.00209753539591, 
0.00452488687783, 0.00486574157506, 0.00850507033039, 0.0101159967629, 
0.0163991223005, 0.0150452373691, 0.0156443766097, 0.0112310639039, 
0.00635593220339, 0.00627766599598, 0.00583041812427, 0.00622371740959, 
0.00624897220852, 0.00420769166036, 0.00305676855895, 0.00291133656815, 
0.00120006857535, 0.00501806503412, 0.00490575781048, 0.00593119810202, 
0.00226874291018, 0.00304999336958, 0.00339087546239, 0.00541958041958, 
0.00445563734986, 0.00431438754455, 0.0038016243304, 0.0037928519329, 
0.00491460867428, 0.00460782305959, 0.00508734881935, 0.00300725278613, 
0.00390896455872, 0.00367811967345, 0.00953591862683, 0.00529614264278, 
0.00243584167029, 0.00427167876976, 0.00291056623743, 0.00227624510607, 
0.00439422473321, 0.00232246538633, 0.00317623830372, 0.00263466042155, 
0.00180200473026, 0.00190912562047, 0.0034896070399, 0.00338638672536, 
0.00548090523338, 0.00697836706211, 0.00720230473752, 0.00746268656716, 
0.00367056664373, 0.0032167269803, 0.00523135203391, 0.00299196443837, 
0.00299119733356, 0.00287306285913, 0.00154657933042, 0.00214861235452, 
0.00163006177076, 0.00157407407407, 0.00137086455858, 0.00124616564417, 
0.000790591955727, 0.00107484854407, 0.00121408336706, 0.00108506944444, 
0.00105398758637, 0.000881834215168, 0.00184409052808, 0.00237529691211, 
0.0013637249172, 0.00190222560396, 0.00264900662252, 0.00156564526951, 
0.00263888888889, 0.00183531139117, 0.00303347280335, 0.0120768352986, 
0.00365330167139, 0.00351443768997, 0.00263080970476, 0.0029703984431, 
0.00265143789517, 0.0014185834431, 0.00150557061126, 0.00144777662875, 
0.00111890957176, 0.000716405690308, 0.000797050911627, 0.000512400081984, 
0.000868526761481, 0.00113392969636, 0.00134609632067, 0.00240013715069, 
0.00128181651712, 0.00110395584177, 0.00156958493198, 0.00208, 
0.00184501845018, 0.00110946745562, 0.000736997262582, 0.00208250694169, 
0.00229084578026, 0.00137639933933, 0.00111462010032, 0.000822518735149, 
0.00200803212851, 0.000987166831194, 0.00041291032964), .Tsp = c(1, 
15.9583333333333, 24), class = "ts")

T3 <- structure(c(0.00192287148809, 0.00149812734082, 0.00192410475681, 
0.00151122625216, 0.00120640491336, 0.00167845582065, 0.00121261115602, 
0.000802568218299, 0.00109170305677, 0.00250626566416, 0.00273597811218, 
0.00242854474127, 0.00160915430002, 0.00124571784491, 0.00192943770673, 
0.00329388800781, 0.00191032700303, 0.00156168662155, 0.00174753289474, 
0.0014917951268, 0.00143639464943, 0.000543773790103, 0.000929525097178, 
0.00141560496294, 0.000966183574879, 0.000719359769805, 0.00190740419629, 
0.00137804317869, 0.00197177251972, 0.001443001443, 0.00203399680372, 
0.00158954433063, 0.00256562068285, 0.00228310502283, 0.00302053966975, 
0.00227352221056, 0.00263239393001, 0.00202608585539, 0.00272386789241, 
0.00269206875129, 0.0027045300879, 0.00276480122033, 0.00405890126487, 
0.00341070582662, 0.00351591413768, 0.00336004135436, 0.00358102059087, 
0.00257289879931, 0.00235733228563, 0.00239624269146, 0.00136103801833, 
0.000862647368926, 0.00145454545455, 0.00168959691045, 0.00246305418719, 
0.0020964360587, 0.00335371868219, 0.00390143737166, 0.00349219391947, 
0.00334507042254, 0.00255102040816, 0.00332922318126, 0.00386753686246, 
0.00246507806081, 0.00432442821449, 0.00312442565705, 0.00408318298357, 
0.00375354756019, 0.00416473854697, 0.00263942103023, 0.0028888688273, 
0.00321817321344, 0.00310218978102, 0.002150738732, 0.00296191819464, 
0.00134732662034, 0.00221708116445, 0.00152797367184, 0.00157932519742, 
0.00220077873709, 0.00207100591716, 0.00260208166533, 0.00310438494373, 
0.00311149524633, 0.00385928454802, 0.00292575886871, 0.00222622707516, 
0.00329074719319, 0.00282614641262, 0.00287542899545, 0.00221198156682, 
0.00311754997249, 0.00315623356128, 0.00287696733796, 0.00296425457716, 
0.00263875450787, 0.00208654631226, 0.00179601096512, 0.00164676821737, 
0.00206262891431, 0.00235895419697, 0.00241963359834, 0.0028610523697, 
0.00516910352976, 0.00160170848905, 0.00254951951363, 0.00275583318023, 
0.00298309579052, 0.00286944045911, 0.00288739172281, 0.00394434096636, 
0.00254428026226, 0.00285214831171, 0.0034924330617, 0.00246440306681, 
0.00266448042632, 0.00389457476678, 0.00253187449136, 0.00171276869059, 
0.00184647850171, 0.00134132164893, 0.00153860077835, 0.000990752972259, 
0.00117518677075, 0.00312927831019, 0.00188867903566, 0.0024, 
0.00269541778976, 0.00263945099419, 0.00242809114681, 0.00378173960022, 
0.00274725274725, 0.00165039196809, 0.00211665098777, 0.00290275761974, 
0.00149017416411, 0.00105244693913, 0.00309917355372, 0.00240432779002, 
0.00297314875035, 0.0015613519471, 0.00196335078534, 0.00227707441479, 
0.00279302706347, 0.00295450068938, 0.00316811446091, 0.00211501661799, 
0.00168990283059, 0.00195694716243, 0.00131815458358, 0.00112343771942, 
0.00214911555629, 0.00157701068863, 0.00171037628278, 0.00230591852421, 
0.00183217295713, 0.00102810143934, 0.00130396986381, 0.00151476899773, 
0.00188470066519, 0.00220449296662, 0.00238267895991, 0.00238639753406, 
0.00147368421053, 0.00113942407292, 0.0018192844148, 0.00152207001522, 
0.00151433207139, 0.00117096018735, 0.000862626698296, 0.00095087163233, 
0.00137000737696, 0.00119202427395, 0.00170319064381, 0.000805585392052, 
0.0012680297987, 0.00189328743546, 0.00186115764005, 0.000719553876597, 
0.000903505601735, 0.000865501125151, 0.00210241778045, 0.00146432374867, 
0.00130625816411, 0.0011895749973, 0.00135374362178, 0.00120192307692, 
0.00160832544939, 0.0015243902439, 0.00240894199268, 0.00218735140276, 
0.00230658337338, 0.00188548179022, 0.0016582220175, 0.00263086274154, 
0.00155166119022, 0.00204834084392, 0.00194670884536, 0.00308959835221, 
0.00154400411734, 0.00152526215443, 0.00343364976772, 0.00269282554337, 
0.00235928547354, 0.00230846919636, 0.00300120048019, 0.00327833023713, 
0.00347844418678, 0.00259690295277, 0.00157392833997, 0.00345536047815, 
0.00336884275699, 0.0023862129916, 0.00216094735932, 0.00478603603604, 
0.00330652368186, 0.00551636824019, 0.00313624204409, 0.00253692126484, 
0.00201631381175, 0.00243072435586, 0.00229410415233, 0.00386954118297, 
0.00298111957602, 0.00305261267732, 0.0038211692778, 0.00334759159383, 
0.00479287915098, 0.0045891294995, 0.00525831471014, 0.00800376647834, 
0.0076613299283, 0.00638604065479, 0.00587868531219, 0.00633955709944, 
0.00453494575849, 0.00617283950617, 0.00314804075884, 0.00425604358189, 
0.00536642629549, 0.00422936152908, 0.00234329232572, 0.00454545454545, 
0.00305280528053, 0.00389501993879, 0.0040267034015, 0.00275554389188, 
0.00409706901986, 0.00506904387345, 0.0065987933635, 0.00594701748063, 
0.00343473994112, 0.00579983814405, 0.00750664048966, 0.00365965233303, 
0.00467423447486, 0.00348250043531, 0.00464471968709, 0.00603621730382, 
0.00358154256205, 0.00445752733389, 0.00501562243052, 0.0035344609947, 
0.00410480349345, 0.00467578297309, 0.00265729470255, 0.00210758731433, 
0.00223771408899, 0.00218998083767, 0.00309374033206, 0.00291738496221, 
0.00184956843403, 0.00297202797203, 0.00329329717164, 0.00318889514162, 
0.00397442543632, 0.00481400437637, 0.002580169554, 0.00440303092361, 
0.00335956997504, 0.00318415000884, 0.00269284225156, 0.00242217637032, 
0.00381436745073, 0.00238326418925, 0.0037407568508, 0.00290474156343, 
0.00335156112189, 0.00227624510607, 0.00376647834275, 0.00223313979455, 
0.00197441840501, 0.00214676034348, 0.00225250591283, 0.00140002545501, 
0.0034896070399, 0.00220115137149, 0.002828854314, 0.00418702023726, 
0.00176056338028, 0.00393487109905, 0.00217939894471, 0.00331724969843, 
0.00234508884279, 0.00282099504189, 0.00239295786685, 0.00269893783737, 
0.00263828238719, 0.00250671441361, 0.00231640356898, 0.00231481481481, 
0.00127947358801, 0.0017254601227, 0.00207530388378, 0.00185655657612, 
0.00131525698098, 0.00227864583333, 0.0018737557091, 0.00220458553792, 
0.00184409052808, 0.00109629088251, 0.00253263198909, 0.00228267072475, 
0.00170293282876, 0.00134198165958, 0.000833333333333, 0.00269179004038, 
0.00198744769874, 0.00209205020921, 0.00146132066855, 0.00113981762918, 
0.00185131053298, 0.00194612311789, 0.00203956761167, 0.00111460127673, 
0.00170631335943, 0.00186142709411, 0.00183094293561, 0.00194452973084, 
0.0014944704593, 0.00153720024595, 0.00184561936815, 0.00151190626181, 
0.000897397547113, 0.00222869878279, 0.00201428309833, 0.00202391904324, 
0.00244157656087, 0.00256, 0.00184501845018, 0.00160256410256, 
0.00115813855549, 0.0016858389528, 0.001741042793, 0.0026610387227, 
0.00167193015047, 0.00201060135259, 0.00219058050383, 0.00233330341919, 
0.000963457435827), .Tsp = c(1, 15.9583333333333, 24), class = "ts")

I know that T1 and T2 are correlated and consider them as ground truth so any distance metric should tell me that (T1, T2) are closer than (T2, T3) and (T1, T3). However, when using dtw in R, I am getting the following:

> dtw(T1, T2, k = TRUE)$distance; dtw(T1, T3, k = TRUE)$distance; dtw(T3, T2, k = TRUE)$distance
[1] 1.107791
[1] 1.568011
[1] 0.4102962

Can someone please explain how to use Dynamic Time Warping for nearest-neighbor queries?

$\endgroup$
  • 1
    $\begingroup$ Could you explain what you mean by a "nearest-neighbor query" in this context and how it is related to dtw? $\endgroup$ – whuber Feb 3 '12 at 15:11
  • $\begingroup$ @whuber: My impression of DTW was that it is a distance metric for time series. And there is this paper indicating that: Faster Retrieval with a Two-Pass Dynamic-Time-Warping Lower Bound by Daniel Lemire et. al with the code provided at code.google.com/p/lbimproved However, I am trying to understand this metric before using it. $\endgroup$ – Legend Feb 4 '12 at 1:31
22
$\begingroup$

Dynamic time warping makes a particular assumption on your data set: one vector is a non-linear time-streteched series of the other. But it also assumes that the actual values are on the same scale.

Lets say you have: $x=1..10000$, $a(x)=1\cdot\sin(0.01*x)$, $b(x)=1\cdot\sin(0.01234*x)$,$c(x)=1000\cdot\sin(0.01*x)$.

Then for DTW, $a$ and $b$ will be extremely similar, while $a$ and $c$ differ almost as much as with Manhattan distance. If you however do a frequency analysis, $a$ and $c$ will be identical with respect to their frequencies, and only differ in magnitude, while $a$ and $b$ have a clearly different frequency.

DTW is not your magic weapon to solve all your time series matching needs. It makes particular assumptions on the kind of similarity you are interested in. If that doesn't match your data, it will not work well. Judging from the data series you shared, you do not need temporal alignment (which DTW does), but actually some appropriate normalization and maybe fourier transformations instead. Treshhold crossing distances might also work well for you, see for example:

  • Similarity Search on Time Series Based on Threshold Queries
    Johannes Aßfalg, Hans-Peter Kriegel, Peer Kröger, Peter Kunath, Alexey Pryakhin and Matthias Renz, EDBT 2006
$\endgroup$
  • $\begingroup$ +1 Thank you for your suggestions. Could you also point me towards some work on Fourier transformations? And finally, I was wondering - are there any practical implementations out there that I can try out? I mean, some databases that actually implement this in action. $\endgroup$ – Legend Feb 4 '12 at 19:14
  • 1
    $\begingroup$ On searching more on this, I came across the SAX symbolic representation work from Keogh et. al of Univ. of Riverside. Would you happen to have any comments on that? $\endgroup$ – Legend Feb 6 '12 at 19:47
  • $\begingroup$ A friend experimented with SAX for motion time series (i.e. motion classification). It did not work for him. That's why I didn't suggest it. Keogh produces papers like crazy, but they aren't very convincing IMHO. He must have proposed at least 10 distance fuctions for time series, that of course all outperform each other. $\endgroup$ – Anony-Mousse Feb 7 '12 at 7:00
  • 2
    $\begingroup$ @Anony I take umbrage with “ Keogh produces papers like crazy, but they aren't very convincing IMHO. He must have proposed at least 10 distance functions for time series, that of course all outperform each other.” I have NOT proposed “least 10 distance functions for time series”. I strongly advocate for 2 distance functions for time series 1) Euclidean distance (ED): two thousand years old 2) DTW: 50 years Those two measures are used in 90% of my papers, and I did not propose or invent either. I have proposed minor changes to both ED and DTW. You say “they aren't very convincing IMHO.”. ... $\endgroup$ – user12039 Jun 16 '12 at 21:51
  • 2
    $\begingroup$ I test with reproducible experiments on every public dataset in the world, and give away all my code. Maybe some folk here are having a hard time using one of my ideas, but more than 2,000 people have successfully used one of my ideas (hit Google up) so maybe the problem is not with the ideas. $\endgroup$ – user12039 Jun 16 '12 at 21:51
4
$\begingroup$

In the 1980s dynamic time warping was the method used for template matching in speech recognition. The aim was to try to match time series of analyzed speech to stored templates, usually of whole words. The difficulty is people speak at different rates. DTW was used to register the unknown pattern to the template. It was called "rubber sheet" matching. Basically you search through some constrained possibilities of how the time series can locally be stretched to optimize the global fit. This approach was shown to be pretty much the same thing as hidden Markov models.

$\endgroup$
4
$\begingroup$

First, you say "dynamic time warping metric", however DTW is a distance measure, but not a metric (it does not obey the triangular inequality).

Paper [a] compares DTW to 12 alternatives on 43 datasets, DTW really does work very well for most problems.

If you want to learn more about DTW, you could glance at Keoghs tutorial http://www.cs.ucr.edu/~eamonn/Keogh_Time_Series_CDrom.zip (warning 500 meg)

The pass is peggy.

There is also a tutorial on SAX http://www.cs.ucr.edu/~eamonn/SIGKDD_2007.ppt

[a] Xiaoyue Wang, Hui Ding, Goce Trajcevski, Peter Scheuermann, Eamonn J. Keogh: Experimental Comparison of Representation Methods and Distance Measures for Time Series Data CoRR abs/1012.2789: (2010)

$\endgroup$
  • $\begingroup$ +1 Thank you so much for your answer. I made corrections to my question. By now, I understand you are a pioneer in time series. It would be great if you have some suggestions on my specific case which I put in one of the comments: The time series data that I have is of an internal twitter-like network and the series itself represents the number of messages generated on a particular topic. I want to find other topics that have a similar timeline as the given one. Thank you once again for your time. $\endgroup$ – Legend Feb 9 '12 at 3:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.