I have a classification problem in which the input feature data are derived from multiple sensors. If the quality of the feature attributes as measured by each sensor varies (for example, because some sensors are more accurate than others, but they measure the same type of data), is there a general approach for how to deal with the difference in quality of the feature data? For example, if I have an estimate of each sensor’s measurement variance is there some way to pre-process the feature data prior to training a classification model that takes into account the sensor’s accuracy? One approach might be to fuse the data from multiple sensors to generate a single training sample for each set of measurements, but I would specifically like to know if there are any other approaches for how to treat feature samples that originate from multiple sources with different measurement uncertainties.
To make this a little more concrete, suppose I am using a sensor to measure two properties of an object (F1 and F2) and I want to classify the object as Class 1 or Class 2 based on those two features. If I only had one sensor (sensor A, very accurate), then I could collect training data using Sensor A and build a classifier. I would normally scale the feature data using Z-scale normalization prior to training the model. I could do the same thing with a 2nd sensor (Sensor B, less accurate than Sensor A). I would expect the classifier built with Sensor B to have a greater test error due to noisier input data.
Now, what if I wanted to use the data from both Sensor A and Sensor B to build a classifier? I am assuming that a model trained on input from both sensors would potentially have lower test error than a model trained using either sensor alone…although I am not sure to what extent this is a reasonable assumption. Let’s assume that a model based on Sensor A or Sensor B alone have X number of training samples and a model built from both A and B have 2X number of training samples. If I know in advance that Sensor A has greater accuracy than Sensor B is there a recommended approach for pre-processing the data from both sensors prior to building a classifier that takes into account the different accuracies? When using data from both sensors I would still scale the (combined) dataset from both sensors. But I am wondering if there is a way to properly treat the different measurement accuracies from both sensors prior to training a model. Perhaps some way to weight the features?
UPDATE: I’d like to also ask a closely related question. In the description above I indicated that different sensors are producing the same features (e.g., F1 and F2). Suppose that I have a new situation in which I have three features (F1, F2, F3). And suppose further that all three are good discriminators for a binary classification problem, but in a real world scenario, the data for feature F3 is noisier than F1 and F2. If I have a good estimate for the measurement variances associated with F1, F2, and F3 is there a way to properly treat these uncertainties prior to training a classification model? In a previous project I have used feature ranking algorithms to select the best N features to use, but in this case I only have a small number of features (< 5) but the quality of the measured data can vary and I am wondering if there are approaches that consider the uncertainty associated with any given feature. Again, for a particular setting I may have access to the estimated variance of any given feature and I would therefore like to know if that can be used to some advantage when training the classifier.