How to approach and model these data - choosing an appropriate model

Question

I have group data where each observation within group is time continuous. (named time in the sample data provided bellow`)

Further, I have price value in given time, appr appreciation/depreciation value, mean1, which is diff(log(mean_values_1) and further mean2 and mean3 which are mean values (mean of price) over different time period. Further, I have variables distance_mean_1_2 and distance_mean_2_3, which are the distances between mean1and mean2 and distance betweenmean2andmean3`.

What I would like to find out is at what level is the dependent variable appreciation, possitive and highest. Or better at which levels is the dependent variable appr maximised.

Also helful would be to find at which levels is the dependent value appr converging to possitive values fastest.

How to approch this. I have long format data. Would I have to turn the data into wide format? So taht each group has one 1 entry per row?

What model (time series?) to use?

I'm using software R.

EDIT: I'm thinking to model each of the independent (explanatory) variable one by one, using survival analysis. Would that be good approach?

EDIT 2: Here is some explanation and hopefully answers the questions raised in answer provided.

1) Why is "convergence" either useful or helpful to know?

The first row in given group is the starting point of measurement. Where appr is always zero. Based on simple command "stop if appr is >100" the measurement process is stopped. The final value is therefore equal or greater than 100. It is desired that the process stops (reaches level 100) as soon as possible. So possbily I misused the word "convergence" here but what I meant is that the appr value reaches the desired value of 100 (or greater) over time as soon as possible.

Based on this I would like to find the appropriate covariate levels that would either maximize the number of possitive values in the given group or reaches the desired appr value of 100 in shorter period of time. So I would like to know at which levels is the best chance of achiving this.

2 ) Issue with variable selection, Which of the predictors are most useful and/or predictive of appr?

It is possible that some of the covariates are not significant in explaning the process of appr. So I would like to find out which covariates are significant. Of course fewer covariates in explaning the maximization or "convergence" would be desired so that the model is parsimonious.

Here is example: samp is the sample data (dput) provided bellow.

samp <- transform(samp, flag=with(samp, ifelse(appr<0, TRUE, FALSE)))

samp$appr <- round(samp$appr, -1)
samp$distance12 <- as.factor(round(samp$distance_mean_1_2, 2))
samp$distance23 <- as.factor(round(samp$distance_mean_2_3, 2))

Model fitted:

fittedsurv <- survfit(Surv(time=samp$time, samp$flag)~samp$distance12)

Here is sample data dput().

structure(list(group = c(1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 2L, 2L, 2L, 2L, 2L, 2L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L), time = c(1L, 2L, 3L, 4L, 5L, 6L, 7L, 8L, 9L, 10L, 11L, 12L, 13L, 14L, 15L, 16L, 17L, 18L, 19L, 20L, 21L, 22L, 23L, 24L, 25L, 1L, 2L, 3L, 4L, 5L, 6L, 1L, 2L, 3L, 4L, 5L, 6L, 7L, 8L, 9L, 10L, 11L, 12L, 13L, 14L, 15L, 16L, 17L, 18L, 19L, 20L, 21L, 22L, 23L, 24L), price = c(1.131975, 1.12895, 1.13415, 1.148075, 1.13455, 1.1477, 1.13155, 1.132425, 1.132075, 1.1337, 1.140325, 1.139275, 1.135375, 1.1411, 1.139725, 1.136825, 1.138125, 1.13345, 1.134025, 1.13605, 1.1198, 1.1196, 1.11835, 1.11765, 1.10785, 1.04955, 1.05675, 1.059725, 1.086375, 1.06595, 1.0822, 1.0889, 1.0833, 1.07315, 1.076325, 1.087975, 1.097175, 1.0922, 1.081475, 1.07805, 1.065925, 1.06045, 1.056725, 1.06545, 1.0684, 1.07615, 1.0807, 1.073775, 1.07355, 1.072425, 1.082375, 1.087875, 1.08915, 1.0981, 1.112825), appr = c(0, 2.67948093361093, -3.8354714984791, -40.1759466933778, 7.02921863293823, -50.3398100548923, 20.3040077769427, 16.4249288032318, 17.9758408232671, 10.7832759989417, -18.3281082147637, -13.736806302254, 3.39095012660988, -21.7115064411535, -15.7055430037954, -2.99078574098922, -8.69851729818822, 11.8884820680219, 9.34723661294901, 0.418115399850166, 72.981782461154, 73.8879957127547, 79.5591719944562, 82.7405717353372, 127.702306268899, 0, 6.81334279630953, 12.4088796621765, 85.6978483488664, 35.695858154697, 110.238403252633, 0, -5.16938982737935, -24.134557144854, -15.2138062388219, 27.7809692318295, 80.0920545947553, 57.681743270464, 8.66871633648522, -7.18890589490256, -64.1461641297459, -90.2918572304211, -108.235349783529, -66.4038669106945, -52.4148259078991, -16.029363936254, 5.08929397612705, -27.1239319224231, -28.1775418005677, -33.4522227661605, 12.8190322208116, 38.0328622314146, 43.8415277969066, 84.2364083416819, 149.282232156898), mean1 = c(0, -0.000125909452303286, 0.000480767625671563, 0.00174112688697965, 0.000197216307187068, 0.00121615479085431, -0.000174056350265378, -8.79262923791369e-05, -9.74266755982556e-05, 1.11326494046937e-05, 0.000374958733439235, 0.000276277046605489, 5.45242034129256e-05, 0.000314368832739276, 0.00022227302140948, 7.9401611874419e-05, 0.000124234137939244, -6.56366814974774e-05, -3.90626893718993e-05, 3.54238189215206e-05, -0.000518055394024303, -0.000486452734504011, -0.000491306161860133, -0.00809928321939048, -0.00388792727271271, -0.0475850358518138, -0.00148495851609912, -0.000766072854589614, 0.00218079377247371, -0.000170151521123416, 0.00124375135307889, 0.020066061096198, -0.000953967459985092, -0.00188206997886047, -0.00283135384960903, 0.000273465517020816, 0.00247455080838277, 0.000367782277040249, -0.000447368358715838, -0.000608727159220307, -0.00226018102656933, -0.00121764245969647, -0.00127198932686792, -0.0125519550862929, -0.00204247878661268, -0.00029715125999033, 0.00203360767447708, 0.000712653091219526, 0.00137546738904835, 0.000213035364990496, 0.000951874176837356, 0.00119929844733625, 0.0065522202860582, 0.00191263631850579, 0.00818748986560495 ), mean2 = c(0, -0.00704195826131931, -0.00627500719528225, -0.00498670169640732, -0.0017388799915012, -0.000170151521123416, 0.00124375135307889, 0.0019985812499813, 0.00153382827948782, 0.0016153716745991, 0.0008664183062876, 0.000786538278354385, 0.00957930055086824, -0.00135328922328085, 0.00106180888463094, 0.000158790614197574, 0.000899337518020904, 0.000367782277040249, -0.000447368358715838, -0.000608727159220307, -0.00127240852760666, -0.00143039032774157, -0.00146627716701067, -0.000856747310940417, -0.000623484107310557, -0.0075061543615107, 0.00111165484916645, 0.00187789269737494, 0.00584401222055937, 0.00765319531854193, 0.00466933643850007, 0.0210108262695513, -0.000278402833809296, 0.000241605996257777, 0.00125095354341931, 0.00318688670096776, 0.00147784903483378, 0.00039605384433157, -0.000632727654295839, -0.000866696154246824, -0.000668468141698117, -0.00108958570139681, -0.00113790320780324, -0.00238803637527786, -0.00112351686458498, -0.0112841232271361, -0.00329564306271869, -0.00225120329744991, 0.00153773392023647, 0.00227537488894973, 0.00161864874852717, 0.00117748472348102, 0.00142481132443494, 0.00119852645921323, 0.00127765117220648), mean3 = c(0, -0.00149598876645349, -0.00441931783382968, -0.0076488409828849, -0.00113846821320018, -0.00163004372170902, -0.00225524378260492, -0.000608260910848812, -0.0023984593618408, -0.00121508194174377, -0.000987657107139134, -0.00478254128195757, -0.00753865101960935, -0.00587250593537236, -0.00213942676875745, -0.0063543836257087, -0.00136629361348317, 0.000197312324906024, -0.000909369742590177, -0.000377384448124907, -0.00682093942827186, -0.00582763856655152, -0.00522235547636016, -0.00112828300967641, 0.000362339162043451, -0.0258114034483171, 0.000480767625671563, 0.00174112688697965, 0.00581915353017895, 0.00395788514502121, -0.00260335810375681, -0.00270979229966919, -0.000160469009492153, 0.000285748828725535, 0.000148759967614098, -5.45749504993476e-05, 2.88816280240856e-05, -0.000231586631810748, -0.000174642513644796, -5.63259988011489e-05, -0.00232587852123847, -0.00827312250145883, -0.00525754196268011, -0.000357446051793123, -0.00196824040279832, -0.00202446609436205, -0.0144237434071857, -0.00690479568146037, -0.0102274404029469, -0.00582252570132376, -0.0101565300751118, -0.00637225436126534, 0.0001992556525688, 0.000682692009971946, 0.00572777812205827), distance_mean_1_2 = c(-0.0365579095629866, -0.0434739583720054, -0.0502297331929546, -0.0569575617763408, -0.058893658075035, -0.0602799643870113, -0.0588621566836718, -0.0567756491413051, -0.0551443941862192, -0.05354015516103, -0.0530486955881793, -0.0525384343564295, -0.043013658008976, -0.0446813160649942, -0.0438417802017707, -0.0437623911994467, -0.0429872878193627, -0.0425538688608303, -0.0429621745301677, -0.043606325508318, -0.0443606786418921, -0.0453046162351342, -0.0462795872402859, -0.0390370513318403, -0.0357726081664306, 0.00430627332387412, 0.00690288668913117, 0.00954685224110112, 0.0132100706891893, 0.0210334175288501, 0.0244590026142726, 0.0254037677876287, 0.0260793324138063, 0.028203008388918, 0.0322853157819468, 0.0351987369658929, 0.0342020351923428, 0.0342303067596357, 0.0340449474640598, 0.0337869784690358, 0.0353786913539018, 0.0355067481122096, 0.0356408342312711, 0.0458047529422805, 0.0467237148643095, 0.0357367428971703, 0.0304074921599643, 0.027443635771299, 0.0276059023024871, 0.0296682418264462, 0.0303350163981333, 0.0303132026742835, 0.0251857937126607, 0.0244716838533682, 0.017561845159964 ), distance_mean_2_3 = c(-0.09726297804778, -0.0958928987336336, -0.0909928132741315, -0.0816028454042668, -0.0802671608838797, -0.0774209623713145, -0.0753397749389736, -0.0748194403205009, -0.0725184076342605, -0.0712921930431132, -0.0699295772025403, -0.0648707588739723, -0.0572775836509485, -0.0510907088828391, -0.0487290090926736, -0.0422952238550939, -0.0408046961036684, -0.0410676451100696, -0.0401973380568562, -0.0397845297898041, -0.0334816457555634, -0.0281404599235143, -0.0234094106090121, -0.0303804108187229, -0.0346306772534791, -0.0564043096569832, -0.0583700357987467, -0.0608772355403157, -0.064515595298027, -0.0686436319641656, -0.0647965225073362, -0.0420206691114687, -0.0428141675619625, -0.0449819863695446, -0.0479621001867693, -0.0476340597192499, -0.0451883905388889, -0.0445890216300379, -0.0448617474751081, -0.0454141486355312, -0.0453484511408505, -0.0382929710990994, -0.0343074184632867, -0.0465019274977842, -0.0465761658815954, -0.0448488510472269, -0.0283914999655629, -0.0207740511928841, -0.00917114340088911, -0.00313558233457029, 0.00797282191737423, 0.0155443747259729, 0.0218973393594655, 0.0231272836679962, 0.0255869954115449)), .Names = c("group", "time", "price", "appr", "mean1", "mean2", "mean3", "distance_mean_1_2", "distance_mean_2_3"), row.names = c(NA, 55L), class = "data.frame")

Answer 1

Some aspects of your question aren't clear. While I think your first concern about the "levels" at which appr is "positive and highest" is clear and addressable, I don't know what is meant by your second question, "at which values appr converging to possitive (sic) values fastest." Perhaps more importantly, why is "convergence" either useful or helpful to know?

You are lucky in that your data isn't large. I wouldn't make this analysis too complicated. There is no reason why a classic ANOVA wouldn't deliver a reasonable answer to your question, in other words, I don't see how survival analysis can help you. Flattening, pushing down the stack or "widening" your file to create a single record would be variance-destroying and not a good idea. You do have an issue with variable selection -- which of the predictors are most useful and/or predictive of appr? I can't imagine that you would need all of them to fit a model, unless there is some theoretical reason that needs to be articulated.

Wrt your first concern about the "levels" which maximize appr, assuming that you are referring to the 3 possible groups, treating group as a qualitative predictor in an ANOVA and, perhaps, dropping the intercept you could obtain coefficients that would be your best indicator of the level where appr is maximized. It would also be worth your while to explore a few of the possible interactions, e.g., price by group, to obtain more representative estimates of the effects.

EDIT 2 Thoughts ************

I see what you mean by convergence now. This is a question that applies to the sequencing of the process. In examining time series line plots containing appr and price for the 3 groups, three things stand out suggesting very different processes at work for each group: first, price for group 2 is much higher than for the other 2 groups, second, price is positively related to appr for groups 2 and 3 and negatively related for group 1 and, finally, I'm left wondering why there is any nonlinear variability in price at all given how it's linearly related to appr in group 2 vs the others. In other words, as the sequence for a process unfolds, why doesn't price change as a linear function for each group instead of only for group 2? That would seem to suggest an immediate solution to the longer sequences for groups 1 and 3 as well as an answer to your question about convergence since the maximal value for appr would be reached with fewer steps.

In terms of variable selection, there are many approaches to that (everybody has an opinion about which approach is best). I rely on the Lasso. You can use one of the widely available R modules for this. On the other hand, the sheer number of possible covariates and their interactions are so few that you can almost do this step by hand and in the absence of some automatic routine. In fact, a careful specification of the predictors and their combinations might lead to better results.