If you're a sport fan, you've probably come across these bogus, cherry-picked stats, things like: "This is the first time a player has scored 32 points on the Ides of March when the temperature was below freezing." You can sense the statistician straining to make something mundane into something unique. But there are other stats which really are intriguing, such as the one (well-known to basketball fans) about LeBron James never posting a 27/7/7 game, even though his career averages are 27.1/7.4/7.3.

I'm curious about how one could create an algorithm that would discover these kinds of statistics, how to quantify the noteworthiness of a given event, and where in the world of math I should start searching for some more information on this topic.

I've got a few ideas, of course...

So in the abstract, we have an event, E, with some set of features (X1...X999) that describe it. We also have a database of all prior Es, and all their features, so whenever we get a new E we can compare it to its predecessors. Our goal is to find some combination of features that will make E unique. Ideally, we want that combination to be as concise as possible. We also would like this stat to be as surprising as possible. (My intuition tells me that a surprising stat is one that's comprised of common events that are somehow not common in conjunction.)

How would we make these concerns legible to a computer? I get the feeling that Bayesian probabilities and information theory (particularly the concept of conditional entropy or mutual information) could help, but I'm not well versed in either of these, and would be curious to know if I'm barking up the wrong tree.

Beyond those theoretical confusions, there's also the question of how to perform the search in some reasonable amount of time, since the solution space is combinatoric. My first thought is to brute force it a bit with a genetic algorithm, but I wonder if there wouldn't be some more elegant, mathematically-rooted procedure. The way I see this, it's like we're trying to play a game of twenty questions in reverse. I know that in a vacuum, the optimal way to play twenty questions is to cut the possibility space in half with each yes or no question. Can that protocol be reversed?

I know this is a fairly vague question (and a few of them at that!) by the standards of SO, but even some vague advice would be appreciated. Thanks for you time!

  • 1
    $\begingroup$ You may wish to read about the Interesting Number Paradox en.wikipedia.org/wiki/Interesting_number_paradox $\endgroup$ – Sycorax Nov 25 '19 at 5:05
  • 1
    $\begingroup$ I've always wondered about the 'Election facts' you sometimes see in articles about US politics. You know the ones. "This is the first election since 1920 where a winning candidate born on an even year lost in Missouri to a challenger with facial hair whose name doesn't end in -son". They have to be made by AI. Please tell me they're made by AI. Please tell me there's not an actual person out there who makes up these facts. $\endgroup$ – Ingolifs Nov 26 '19 at 5:14
  • 1
    $\begingroup$ Surprise: See stats.stackexchange.com/questions/66186/… and ilab.usc.edu/surprise $\endgroup$ – kjetil b halvorsen Nov 26 '19 at 5:57
  • 1
    $\begingroup$ re election facts: xkcd.com/1122 $\endgroup$ – shimao Nov 26 '19 at 7:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.