I have a dataset of case studies like that:

Gender ; Age ; Symptom ; Condition

M ; 20 ; Fever ; Flu
F ; 29 ; Arm pain ; Trauma
F ; 40 ; Fever ; Cancer
M ; 90 ; Diarrhea ; Flue

and a model that predict the condition of each new sample (Gender + Age + Symptom). Due to the fact that each condition have many different symptoms, i would like to build an algorithm that for each sample + prediction , will give me the best symptoms that i need to check with the person to increase the certainty of the model prediction or update the prediction to other condition .

i have checked some exotic approaches such as active learning or cost sensitive learning but i'm pretty sure that there are some more basic approaches that i'm missing here.

  • $\begingroup$ I think the simplest solution might be a lookup table: for each condition, rank the symptoms in order of prevalence or informativeness, and iterate between prediction with the model trained on the data and collecting additional information from the table. If you however want rigorous evaluation of probability, then you need to start with a probabilistic model for symptoms (and symptom checking) and diagnosis/disease confirmation. I'd have to think about what such a model would be. $\endgroup$ – deasmhumnha Aug 22 '18 at 20:06
  • $\begingroup$ I need it in real time , so its an issue to check for each sample all the possible models .. can you elaborate more about the probabilistuc model? Thanks $\endgroup$ – Latent Aug 23 '18 at 4:36
  • $\begingroup$ You should look into latent variable models. Here the latent variables would be possible but unconfirmed symptoms. You could then approximate the probability distribution over the latent variables given validated symptoms $X$ and confirmed diagnoses $Y$ as $P(Z |X,Y)$ either using EM or a Bayesian approach. I'll try to write a formal description of such a model later. I also remember reading a paper about imputing missing data in SVMs based on the similar idea, though in your case the focus is on the latent variables. $\endgroup$ – deasmhumnha Aug 23 '18 at 20:59

Following my comment above, if $X$ is our confirmed symptoms (as binary present/absent variables) and $Y$ is our diagnosis (as one-hot vector/categorical variable), the $Z$ is our untested/unconfirmed systems (also as binary variables). The simplest model would be multi-logistic regression such that $E[Y|X,Z]=\mathop{\mathrm{softmax}}([X,Z]\theta)$, but we could choose our estimator to be any function $f$ such as a neural network. Here $[X,Z]$ is sample-wise concatenation. To fit this model, we use the likelihood that $$\mathcal{L}(Y|X,Z)=\sum_i \mathrm{Cat}(Y=y_i|E[Y]=f(\theta,x_i,z_i))$$

For the frequentist solution, we use EM algorithm and any associated shortcuts or heuristics (a discussion of which is outside the scope of this post). Once we have our estimator for $\theta$ we can find the most likely $z$ given $x$ and/or $y$ and check for those symptoms with a value of 1.

A Bayesian approach would instead generate many samples of both $\theta$ and $Z$ given the above likelihood and reasonable priors. We would then take the expectations $E[Z|X=x, Y=y]$ and/or $E[Z|X=x]$ over all values of $\theta$. It might be worth it to use variational Bayes however if your training data is particularly large. You could then choose to check for symptoms with highest expectation (highest marginal probability).

Updating our diagnosis/prediction is simply running our updated vector through the model, which should take only a few microseconds depending on the model. Updating the model itself, however, requires refitting and is therefore only recommendable after collecting a substantial number of new examples.

Finally, as your model incorporates more computer-aided diagnoses, you also might want to maintain that your model isn't substantially different from a human expert and you could probably use a discriminator from adversarial networks for that purpose.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.