Multilayer perceptrons are great for discovering associations between variables defining independent events based on the same underlying associations in reality.

Less cryptically put, MLP's are great for learning, for example, whether a tumor is cancerous based on its size and density, because these features are inherently associated with properties relating to cancer formation, and thus are good predictors of whether or not a given tumor is cancerous.

But cancerous tumors are independent events, e.g. if Bob's tumor is cancerous, that fact does not make it any more likely that Alice's tumor is also cancerous, it simply reveals more information about the underlying pattern that can be trained upon without the events themselves having any actual interdependence. What if the problem at hand does not involve independent events, but dependent ones. For example, what if I wanted to predict earthquakes?

Earthquakes are dependent events, as an earthquake here creates and relieves stresses in the earth there, actually changing the likelihood that a future earthquake will occur at a given time.

Now convention would dictate that for predicting such a problem, I start with an LSTM or some other type of RNN, so I can express the ordinality of earthquake events through the directed graph itself, rather than though the data. But earthquake events really aren't a matter of "is there an earthquake happening here and now, or isn't there" because earthquakes happen constantly, and the vast majority are too weak to detect, and thus too weak to be present in the dataset.

I'm trying to overcome this factor of — and excuse my crude use of mathematics terminology from a different field — a "sparse" dataset over the true set of events, by using a model not inherently structured toward ordinal data. Is a basic multilayer perceptron well-suited toward this type of dependent-events problem, or is there a clearly better structure? Also, if you know something I clearly don't, please share. I'm here to learn :)


To answer the question in the title:

Are basic multilayer perceptrons well-suited to prediction of non-independent events?

Yes they are. The behavior you describe, the probability of a current or future occurrences of an event being dependent on the past occurrences of the event, can be described as auto-regressive behavior. And for this MLP can be used (see for example the NNETAR package in R).

To your point, MLP are not structured to take ordered or sequential data, so for that you need to format your input data in a specific way. See here for details.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.