While casually reading some mass market works on chaos theory over the last few years I began to wonder how various aspects of it could be applied to data mining and related fields, like neural nets, pattern recognition, uncertainty management, etc. To date, I've run into so few examples of such applications in the published research that I wonder if a) they've actually been put into practice in known, published experiments and projects and b) if not, why are they used so little in these interrelated fields?
Most of the discussions of chaos theory I've seen to date revolve around scientific applications that are entirely useful, but have little to do with data mining and related fields like pattern recognition; one of the archetypical examples is the Three-Body Problem from physics. I want to forego discussion of ordinary scientific applications of this kind and restrict the question solely to those applications which are obviously relevant to data mining and related fields, which seem to be few and far between in the literature. The list of potential applications below can be used as a starting point of a search for published research, but I'm only interested in those applications that have actually been put into practice, if any. What I'm looking for are known implementations of chaos theory to data mining, in contradistinction to the list of potential applications, which is much broader. Here's a small sampling of off-the-cuff ideas for data mining applications that occurred to me while reading; perhaps none of them are pragmatic, perhaps some are being put to practical use as we speak, but go by terms that I'm not yet familiar with:
- Identifying self-similar structures in pattern recognition, as Mandelbrot did in a practical way in the case of error bursts in analog telephone lines a few decades ago.
- Encountering Feigenbaum's Constant in mining results (perhaps in a manner similar to how string theorists were startled to see Maxwell's Equations pop up in unexpected places in the course of their research).
- Identifying the optimal bit depth for neural net weights and various mining tests. I wondered about this one because of the vanishingly small numerical scales at which sensitivity to initial conditions comes into play, which are partially responsible for the unpredictability of chaos-related functions.
- Using the notion of fractional dimensions in other ways not necessarily related to fascinating fractal curiosities, like Menger Sponges, Koch Curves or Sierpinski Carpets are. Perhaps the concept can be applied to the dimensions of mining models in some beneficial way, by treating them as fractional?
- Deriving power laws like the ones that come into play in fractals.
- Since the functions encountered in fractals are nonlinear, I wonder if there's some practical application to nonlinear regression.
- Chaos theory has some tangential (and sometimes overstated) relationships to entropy, so I wonder if there's some way to calculate Shannon's Entropy (or limits upon it and its relatives) from the functions used in chaos theory, or vice versa.
- Identifying period-doubling behavior in data.
- Identifying the optimal structure for a neural net by intelligently selecting ones that are most likely to "self-organize" in a useful way.
- Chaos and fractals etc. are also tangentially related to computational complexity, so I wonder if complexity could be used to identify chaotic structures, or vice-versa.
- I first heard of the Lyapunov exponent in terms of chaos theory and have noticed it a few times since then in recipes for specific neural nets and discussions of entropy.
There are probably dozens of other relationships I haven't listed here; all of this came off the top of my head. I'm not narrowly interested in specific answers to these particular speculations, but am just throwing them out there as examples of the type of applications that might exist in the wild. I'd like to see replies that have examples of current research and existing implementations of ideas like this, as long the applications are specifically applicable to data mining.
There are probably other extant implementations I’m not aware of, even in areas I'm more familiar with (like information theory, fuzzy sets and neural nets) and others I those I have even less competence in, like regression, so more input is welcome. My practical purpose here is to determine whether or not to invest more in learning about particular aspects of chaos theory, which I'll put on the back burner if I can't find some obvious utility.
I did a search of CrossValidated but didn't see any topics that directly address the utilitarian applications of chaos theory to data mining etc. The closest I could come was the thread Chaos theory, equation-free modeling and non-parametric statistics, which deals with a specific subset.