0
$\begingroup$

I have some signal data of a robot recorded in every minute each day.

e.g.,

42.27,42.61,45.01,48.50,,50.82,50.30,49.57,46.24,50.12,48.39,48.73,45.08,45.81,46.86,46.51,45.16,42.36,42.61,41.10,40.91,40.65,40.82,40.88,40.12,39.33,39.79,39.80,36.95,36.89,36.30

However, I want to look at a more global patterns of the timeseries. With regard to this, I considered the moving average of the timeseries that erases short-term fluctuations. However, I would like to know if there are any other techniques in time-series analysis that I can use to get global patterns?

I am happy to provide more details if needed.

EDIT:

Suppose I have three timeseries as follows.

T1 = [0.6619346141815186, 0.7170140147209167, 0.692265510559082, 0.6394098401069641, 0.6030995845794678, 0.6500746607780457, 0.6013327240943909, 0.6273292303085327, 0.5865356922149658, 0.6477396488189697, 0.5827181339263916, 0.6496025323867798, 0.6589270234107971, 0.5498126149177551, 0.48638370633125305, 0.5367399454116821, 0.517595648765564, 0.5171639919281006, 0.47503289580345154, 0.6081966757774353, 0.5808742046356201, 0.5856912136077881, 0.5608134269714355, 0.6400936841964722, 0.6766082644462585]

T2 = [0.8465140461921692, 0.8005352020263672, 0.7825719118118286, 0.8457388281822205, 0.7688464522361755, 0.7278903126716614, 0.7305904030799866, 0.7086656093597412, 0.5729755163192749, 0.6853111386299133, 0.7004281282424927, 0.7262798547744751, 0.7586643099784851, 0.7935684323310852, 0.6071596145629883, 0.5113456845283508, 0.48870643973350525, 0.5383914709091187, 0.5680867433547974, 0.5676274299621582, 0.576948881149292, 0.5799109935760498, 0.6209957003593445, 0.6316736340522766, 0.5946987867355347]

T3 = [0.6238030791282654, 0.6118258833885193, 0.5825846791267395, 0.5882349610328674, 0.5327020883560181, 0.6147461533546448, 0.5573892593383789, 0.5562931895256042, 0.587559163570404, 0.5538188815116882, 0.5504220724105835, 0.5547892451286316, 0.5519694685935974, 0.5377979278564453, 0.5907807350158691, 0.5775166749954224, 0.6759968996047974, 0.6989523768424988, 0.6136959195137024, 0.7389665842056274, 0.7068051099777222, 0.7298943400382996, 0.7541239857673645, 0.7532614469528198, 0.7781376242637634]

I calculated the loess smoothing for the three timeseries using 1/3 as the span. The results I got is as follows.

T1_smoothed = [0.69572856082008, 0.6817093384338818, 0.6678761614398027, 0.654927768075122, 0.6386380982668204, 0.6246523007337748, 0.6183557694682981, 0.6169497177525506, 0.6143970451068759, 0.6176077395281502, 0.621926617777711, 0.613623398511036, 0.5930251821144936, 0.5717815504920749, 0.5473504652036533, 0.5240158001390338, 0.5159032563798367, 0.5259440685482747, 0.5397566873135351, 0.5529951092276975, 0.5691012445638404, 0.5898740543670239, 0.6112994750567291, 0.6322335772994212, 0.6549343072354231]

T2_smoothed = [0.8335640912677699, 0.8162743485734795, 0.7984554023172393, 0.7798181985514753, 0.7634474378493065, 0.7466269682786937, 0.7266432234420882, 0.713488303029767, 0.7071368904348893, 0.7046679293979241, 0.7071621856527248, 0.6981979356910865, 0.6684702831566509, 0.6354187588989586, 0.5991178757417265, 0.5636139184039238, 0.5512163943659039, 0.5463140255144258, 0.5525813451015571, 0.5670901734823733, 0.5812269104698217, 0.5934860185032088, 0.6016378593194055, 0.6101734957850382, 0.6185684605419651]

T3_smoothed = [0.6218908016723772, 0.6085510050875651, 0.5958672378501175, 0.5841018335458528, 0.5756731334524192, 0.5692939653652245, 0.563300597871705, 0.5594267837914737, 0.5581806196621288, 0.5570719072557064, 0.5552750403932354, 0.5550768641410069, 0.561760569494235, 0.5782866222244073, 0.6043067376386835, 0.6331067592241593, 0.6676800863978486, 0.6874843695435778, 0.7001699973976037, 0.710547032141524, 0.7201562741121644, 0.7310730887608722, 0.7455783201097064, 0.7606993864574407, 0.7758317090246474]

Next I tried to calculate the residuals of each timeseries. The residuals of the three timeseries are as follows.

T1_residuals = [0.0337939466385615, -0.03530467628703493, -0.02438934911927937, 0.015517927968157874, 0.03553851368735261, -0.025422360044270897, 0.017023045373907197, -0.010379512555982129, 0.02786135289191005, -0.030131909290819547, 0.039208483851319365, -0.03597913387574381, -0.0659018412963035, 0.02196893557431978, 0.06096675887240022, -0.012724145272648313, -0.0016923923857272705, 0.008780076620174104, 0.06472379151008356, -0.055201566549737846, -0.011772960071779748, 0.004182840759235806, 0.050486048085293556, -0.007860106897050967, -0.021673957210835426]

T2_residuals = [-0.01294995492439932, 0.01573914654711228, 0.015883490505410713, -0.06592062963074519, -0.005399014386869072, 0.018736655607032304, -0.003947179637898368, 0.004822693670025746, 0.13416137411561435, 0.01935679076801078, 0.0067340574102321415, -0.02808191908338864, -0.0901940268218342, -0.1581496734321266, -0.008041738821261801, 0.052268233875573, 0.06250995463239861, 0.007922554605307108, -0.015505398253240266, -0.0005372564797848645, 0.004278029320529675, 0.013575024927159007, -0.019357841039938983, -0.0215001382672384, 0.023869673806430458]

T3_residuals = [-0.0019122774558881783, -0.003274878300954187, 0.013282558723378024, -0.004133127487014665, 0.04297104509640115, -0.0454521879894203, 0.005911338533326149, 0.003133594265869455, -0.029378543908275212, 0.003253025744018201, 0.00485296798265189, 0.0002876190123752842, 0.009791100900637617, 0.040488694367961986, 0.013526002622814337, 0.0555900842287369, -0.00831681320694877, -0.011468007298920946, 0.08647407788390127, -0.028419552064103404, 0.013351164134442262, 0.00117874872257262, -0.00854566565765813, 0.007437939504620905, -0.002305915239116052]

Then I got median of each residual timeseries.

T1_median = -0.0016923923857272705
T2_median = 0.004278029320529675
T3_median 0.003133594265869455

My first question is whether my understanding of how to get the residual median is correct?

I am willing to do the above described procedure for 1/48 to 2/3 of my three timeseries. Then, I would like to perform the elbow method that you have suggested. My next question is that since I would like to identify the point to stop smoothing that is good for three of my timeseries would it be fine if I considred the mean of the three median residula values? or is there a better way of doing this?

I look forward to hearing from you.

Improve this question
$\endgroup$
8
  • 2
    $\begingroup$ Please tell us more about what you mean by a "global pattern" in the series and what constitutes "looking" at it. $\endgroup$
    – whuber ♦
    Commented Mar 31, 2020 at 13:18
  • $\begingroup$ @whuber Thanks a lot for the comment. I am more interest about longer-term trends in the timeseries (i.e. perhaps hourly basis instead of minutes). For that purpose I am currently using moving average. However, I thought that there are more interesting techniques I can use for this instead of moving average. Please kindly let me know your thoughts. Thank you :) $\endgroup$
    – EmJ
    Commented Mar 31, 2020 at 14:02
  • 2
    $\begingroup$ Perhaps you mean an exploratory analysis somewhat in the spirit of stats.stackexchange.com/a/299480/919? $\endgroup$
    – whuber ♦
    Commented Mar 31, 2020 at 15:00
  • $\begingroup$ @whuber Thanks a lot for the great answer. This is exactly what I was looking for. I have three questions that I would like to get your feedback on; 1) what is the technique you used for smoothing (I haven't used R before, so, what is like the equivalent package that I can use in python instead of loss)?, 2) I love your idea of spread. I would also like to draw one for my dataset. However, I am not clear what is the grey area near the line in your plot is? So the line represents the median of the smoothed weights? $\endgroup$
    – EmJ
    Commented Apr 1, 2020 at 0:15
  • 1
    $\begingroup$ In the paragraph beginning "This plot shows a clear "elbow"," I explain what the elbow means. The plots were drawn using the ggplot2 package for R. I am not familiar with graphics packages for Python, but keywords you could use to search for comparable ones would be "ggplot" and "grammar of graphics." $\endgroup$
    – whuber ♦
    Commented Apr 1, 2020 at 12:55

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.