1
$\begingroup$

I am a first-year grad student in Computer Science, and I need some help with a problem that I think is statistically oriented. I have taken a statistics course, but it was abysmal and I haven't had time to rectify that. But anyway, my problem stems from a project I'm working on involving genetic programming, where I'm randomly generating functions. Please bear with my description, as it's been a while since I've had a formal theory course too.

I have two continuous (but not onto) functions F and G, both of which map N variables to a single output. The domain of the input variables is the integers between -100 and 100. The range of the output is the Real numbers. I want to find some statistical measure of how "similar" the two functions are; given the finite inputs (of which there will be 201^N possible), how much variance(?) there is between the two functions outputs. Two identical functions should return no variance, and two wildly different functions should return a high variance.

Since N will typically be greater than 6, I can't iterate through all the possible inputs and compare the outputs, so I figured I could take some sampling at regular intervals (e.g. every multiple of 10, so that it's only 10^N). But here's about where I realize I have no idea what I'm doing. How do I determine if two numbers are "highly variant" from each other? What sample size do I need to use to have confidence in my results?

My current approach is to compare the functions with a two-sided Kolmogorov-Smirnov Test. Since that test doesn't seem to scale well to multi-variate problems, I've taken advantage of my limited domains to just treat the problem as having a single variable by concatenating my variables. So the first value of the variable is (-100:100:100:100:100:100), the second is (-100:100:100:100:100:099), and the last is (100:100:100:100:100:100). Does that even make sense?

Improve this question
$\endgroup$
2
  • $\begingroup$ Would something like summing the squares of the differences between F and G (or the absolute value of their differences) over all the values for which you have outputs for both suffice as a measure? With 6+ dimensions, I'd suggest not using a lattice-type grid of values, by the way, but random draws from the space of values (though you might use quasi-random sequences instead I suppose). $\endgroup$
    – Glen_b
    Commented Oct 12, 2013 at 22:16
  • 1
    $\begingroup$ Have you considered Kullback-Leibler and Bhattacharya distances? $\endgroup$
    – user92883
    Commented Oct 23, 2015 at 0:46

1 Answer 1

Reset to default
-1
$\begingroup$

Calculate a correlation of two functions over a set of random examples. The two-sided Kolmogorov-Smirnov test compares one-dimensional distributions, not multidimensional functions.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Correlation is useless for this purpose, even with single-predictors. Consider X on 1,2,3,4,5 and fits f and g being (12, 15, 17, 19, 23) and (101.12, 101.15, 101.17, 101.19 and 101.23) respectively. They're perfectly correlated, but they're nowhere near each other. $\endgroup$
    – Glen_b
    Commented Oct 12, 2013 at 22:01
  • $\begingroup$ acbart writes: I want to find some statistical measure of how "similar" the two functions are. The two functions are "similar". $\endgroup$
    – user31264
    Commented Oct 13, 2013 at 0:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.