I've got an introductory AI course in my university, and I was taught about decision trees. I'm now facing a classification problem that seems solvable with a DT, but I'm stuck with an unseen situation.

Let's say that I'm trying to mimic a function using data from many experimental results, for each outcome my instrument records not only the result but also the reliability, expressed as a probability value.

An extract of my data set looks like this:

inputs |  out  | reliability
89,'g' | true  | 0.8
89,'g' | true  | 0.75
89,'g' | false | 0.2
89,'g' | false | 0.1
89,'g' | false | 0.13

What I want to achieve is the creation of a model able to replicate the behavior captured by the instrument, generating the outputs coherently with the given reliability.

Is using a decision tree a bad idea? Otherwise, how should I treat the information about the reliability?

  • $\begingroup$ Is out obtained from reliability, by something like: if reliability <= 0.5 then out=true, else out = false? Because in that case you could model reliability directly and ignore out. $\endgroup$
    – utobi
    Feb 5 '19 at 12:48

One way you can approach this is to use a classification algorithm that allows weighting of the instances in the dataset. Some implementations of some algorithms may support this directly, but if the specific implementation you're using doesn't, you can do it yourself by sampling the dataset based on the reliability.

For example if your dataset is relatively small you could oversample by simply replicating each point floor(10 * r) times where r is the reliability - so 0 ≤ r < 0.1 would get no additional replicates, 0.1 ≤ r < 0.2 would get one, and so on.

If you have a large dataset you might prefer to under- and/or oversample instances at random with a probability derived from the reliability, so that low-reliability points were less likely to be included, and so on.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.