Skip to main content
replaced http://stats.stackexchange.com/ with https://stats.stackexchange.com/
Source Link

For neural networks, my initial gut feeling was that you could simply modify the error or discrepancy function to include a class-specific penalty. For example, suppose you're using the typical minimize-the-sum-squared-error approach, you normally minimize $\sum_i(y_i - o_i)^2$, where $o$ is the network's output and $y$ is the "true" label for example $i$. You could simply scale that by a constant that depends on the true and predicted class. Kukar and Kononenko (1998) looked at a few other approaches and found that this one typically works best.

Cost-sensitive random forests shouldn't be a problem either; they were (briefly) discussed in this threadthis thread.

There are about a zillion random forest and neural network implementations floating around though, so it's hard to know if these options have been added to your software package of choice.

For neural networks, my initial gut feeling was that you could simply modify the error or discrepancy function to include a class-specific penalty. For example, suppose you're using the typical minimize-the-sum-squared-error approach, you normally minimize $\sum_i(y_i - o_i)^2$, where $o$ is the network's output and $y$ is the "true" label for example $i$. You could simply scale that by a constant that depends on the true and predicted class. Kukar and Kononenko (1998) looked at a few other approaches and found that this one typically works best.

Cost-sensitive random forests shouldn't be a problem either; they were (briefly) discussed in this thread.

There are about a zillion random forest and neural network implementations floating around though, so it's hard to know if these options have been added to your software package of choice.

For neural networks, my initial gut feeling was that you could simply modify the error or discrepancy function to include a class-specific penalty. For example, suppose you're using the typical minimize-the-sum-squared-error approach, you normally minimize $\sum_i(y_i - o_i)^2$, where $o$ is the network's output and $y$ is the "true" label for example $i$. You could simply scale that by a constant that depends on the true and predicted class. Kukar and Kononenko (1998) looked at a few other approaches and found that this one typically works best.

Cost-sensitive random forests shouldn't be a problem either; they were (briefly) discussed in this thread.

There are about a zillion random forest and neural network implementations floating around though, so it's hard to know if these options have been added to your software package of choice.

Source Link
Matt Krause
  • 21.4k
  • 3
  • 68
  • 111

For neural networks, my initial gut feeling was that you could simply modify the error or discrepancy function to include a class-specific penalty. For example, suppose you're using the typical minimize-the-sum-squared-error approach, you normally minimize $\sum_i(y_i - o_i)^2$, where $o$ is the network's output and $y$ is the "true" label for example $i$. You could simply scale that by a constant that depends on the true and predicted class. Kukar and Kononenko (1998) looked at a few other approaches and found that this one typically works best.

Cost-sensitive random forests shouldn't be a problem either; they were (briefly) discussed in this thread.

There are about a zillion random forest and neural network implementations floating around though, so it's hard to know if these options have been added to your software package of choice.