0
$\begingroup$

I created an algorithm to fill missing values in a list of numbers. I have a list for example [2,4,5,3,6,8,4,NaN,5,NaN,4,6,4,2,4,6,NaN.....]. Think of it much much bigger.

I am looking for tools where they can do the same thing. Fill those missing values. But not in random. In my case, I use a neural net to find the most likely number to put there. I have 90% correct with my algorithm and I want to compare it with other tools in the same data.

Can you suggest any tool to try it?

$\endgroup$
6
  • 5
    $\begingroup$ Methods of interpolation have been used for this over several centuries. Popular methods include nearest neighbour, linear, cubic, cubic spline and piecewise cubic Hermite. mathworks.co.uk/help/matlab/ref/interp1.html is a handy overview, but implementations are by no means restricted to MATLAB. $\endgroup$
    – Nick Cox
    Commented Jun 26, 2013 at 15:19
  • $\begingroup$ It's not entirely clear to me what you are asking. Do you want to know about other interpolation algorithms so you can see if yours works better or worse? $\endgroup$ Commented Jun 26, 2013 at 15:36
  • $\begingroup$ exactly. But not methods. I want tools, where I can import my data and see if they have better results than mine $\endgroup$
    – Tasos
    Commented Jun 26, 2013 at 15:40
  • $\begingroup$ This problem in general has been extensively studied. The results uniformly are that the quality of the predictor ("interpolator") depends on the statistical nature of the data. Some predictors work better with certain kinds of data, some with other kinds of data; no one works universally well. Thus, something to think about is whether you want to compare your predictor to others only for your kind of data or if you want to compare it to other predictors for more general data. $\endgroup$
    – whuber
    Commented Jun 26, 2013 at 16:26
  • $\begingroup$ It is only for my own data. Not for general uses. That's why I needed few tools, to try them all. I will try the tools on the answer bellow as soon as I will have access to my pc where I have installed the R package. $\endgroup$
    – Tasos
    Commented Jun 26, 2013 at 16:59

1 Answer 1

3
$\begingroup$

You're not going to find a piece of software that only interpolates, so you're going to have to get something like Matlab, R, etc, and perhaps also additional toolkits or packages (for Matlab and R, respectively).

If you get R, base interpolation methods include approx (linear) and spline (cubic, Hermite). R package akima has aspline (a unique algorithm that results in "hand drawn" results), package signal has interp (filter-based) and interp1 (linear, nearest-neighbor, pchip, cubic, cubic spline with smoothing), package splines has interpSpline (B-spline).

You may need to slightly reformulate your data to use some of them, giving an x (position of the value in the series) and a y (value) for non-NaN's, then asking them to interpolate at the x values that had NaN's.

That's just what I found by searching for 'interpolate'. R has many other packages to do neural nets, wavelets, time series, and a whole host of other methods that would interpolate your data. Some of these other methods may require some domain knowledge (what kind of data series this is).

So it might be wise to first figure out methods that can do interpolate-ish things (NN, state space, time-series methods, wavelets, splines, etc, etc) and then find a package that does many of them and proceed from there.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.