I'm doing a task where I need to work with healthcare data from a few different sources. For example, one is an audio signal recording while another is biometric signal reading such as ECG.

Both of these will be available for each patient, and we are trying to do some machine learning classification to do some predictive diagnosis. In fact, we have created a multiclass classifier model of the audio signal, as well as the ECG data.

Now, we are trying to combine the two signals modalities into one single, better model. What would be the best approach for this kind of problem? We are researching multimodal ML algorithms but would like to get a feel of what is the best way to go about creating this model.


1 Answer 1


I have not worked with multimodal ML, but from my experience I would try two different approaches to your problem:

  1. Feed a machine learning algorithm with all the data, prior to data normalize the inputs, your algorithm, e.g. a SVM, should be able to learn the relations between different sources.
  2. Make one ML algorithm for each signal source, and combine the outputs of these algorithms in a new one, however this a bit more tedious a error prone.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.