I have an interesting modelling problem in which I am trying to forecast the occurrence of a type of weather event using an empirical model driven by measurements of a number of different physical quantities in different locations.
There are a number of complications. First, the predictor measurements are not independent in the sense that the driving force that generates these weather events (which we don't measure directly) has an impact on all the quantities that we measure, driving them from their normal values and giving us warning of the weather event to follow.
The second complication is that the training data is relatively sparse, so it cannot support highly complex models. Using simple linear models is problematic though, because of the stated non-independence of the predictors. We could use an approach like principal component regression to solve this, except for the third complication, which is that for both the training data we have and in operations, data is frequently missing due to sensors being broken or offline at the time.
So, what strategies might be useful here? There are a few basics approaches I can see. Say for example we want to forecast the probability of event F occurring, based on the unreliable predictors A and B. We could train 3 completely separate models:
$$P(F|AB), P(F|A), P(F|B)$$
using whatever approach works best for each model and then use whichever model is appropriate given the available data. This however does not generalise for many predictors (we have at least 50 different quantities we could use) due to the curse of dimensionality.
The next simplest approach I've considered would be to construct separate models for each predictor alone
$$ P(F|A), P(F|B) $$
and then learn weights for each model so that we have $$ P(F|AB) = \frac{w_A P(F|A) + w_B P(F|B)}{w_A + w_B} $$
the weights would be learned over the whole training set, setting the relevant weight to zero for each instance where a predictor is missing. This is a much more tractable solution, but I fear it would miss important interactions between the predictors and would sacrifice too much performance for the gain in simplicity.
A third approach I can vaguely conceive of but I'm not sure if it is valid, is to try and express
$$P(F|AB)$$
in terms of the individual models
$$P(F|A), P(F|B) $$
and the conditional probabilites e.g.
$$P(A|B)$$
the form of which would need to be learned from the training data. I haven't managed to come up with a useful form for this yet, although maybe it isn't possible (or advisable). The curse of dimensionality is likely to be a factor here as well I guess.
I realise this could be phrased as a data imputation problem but I'm looking for a solution that gives a good trade-off between practicality and performance.
