3
$\begingroup$

I have a situation where I need to calculate the speed of a vehicle for a given time period. The data I'm working with is very spotty in the temporal dimension— we might have 5-10 speeds recorded within seconds of each other, but other speeds in the set might be many seconds or minutes apart. The speeds can sometimes be wildly inaccurate which is why we're using median instead of mean.

What we're noticing is this: because of the spottiness of the data, clusters of closely recorded points tend to skew the results of a median calculation because they take up a large amount of the sorted set from a count standpoint, but only represent a relatively small amount from a temporal standpoint.

We've considered interpolating the points (using a linear function) between large gaps in our measurements in order to smooth it out.

Question: is there a better way to solve this problem than by interpolating missing points? Is there a better option than using median (assuming we'll have outliers)?

Improve this question
$\endgroup$
5
  • $\begingroup$ How dynamic do you expect your speed to be? Do you have better information on your acceleration? (Speed is a simple function of acceleration and time.) Could you pull a sample of your data based upon defined time intervals? $\endgroup$
    – Tavrock
    Commented Apr 10, 2017 at 18:37
  • $\begingroup$ Unfortunately that's the problem— I can't get the data for a set time interval. I'm working with data that comes from a GPS that has spotty connectivity. There will be moments where it has a great connection and plenty of regularly spaced points, but other times it might have a bad connection and not update itself for long periods of time. $\endgroup$
    – Jim Heising
    Commented Apr 11, 2017 at 16:48
  • $\begingroup$ I think that this problem may have many solutions. Perhaps the best should be to use the Discrete Fourier transform. $\endgroup$
    – Rafael Marazuela
    Commented Jun 29, 2019 at 7:09
  • $\begingroup$ I answered a similar question at stats.stackexchange.com/questions/504734/… $\endgroup$
    – user225256
    Commented Jul 14, 2022 at 20:25
  • $\begingroup$ "The data I'm working with is very spotty in the temporal dimension" can you tell more about the data gathering process. Why is it spotty? If it correlates with the speed, e.g in tunnel where a vehicle has to drive more slowly there is no connection, then you have a problem with sample bias. Or can we assume that the sampling is random? $\endgroup$
    – Sextus Empiricus
    Commented Apr 23, 2023 at 19:37

2 Answers 2

Reset to default
1
$\begingroup$

If the data is static I would treat those repeated measures in a few seconds as a repeated measurement. I would take the mean of those values and use that with all the other points.

If the data is dynamic you could take the median of the spots at fixed intervals before calculating the average speed.

Improve this answer
$\endgroup$
1
$\begingroup$

Here are three possible approaches:

  1. You could compute a median of means, counting clusters as single points by taking the mean of clusters as a single point.

  2. You could compute the empirical distribution function along with a confidence interval. Since your measurements have errors, you could also estimate the error in the estimated median with a Monte Carlo approach.

  3. (Possibly simpler) You could compute a mean of means, which allows an easier estimation of the error.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.