1
$\begingroup$

I have a big dataset with around 100k samples and 2k real-value features. The target variable is in [-1,1], but its distribution is highly concentrated around zero (around %50 of the values are in [-0.01,0.01]. This is its histogram: enter image description here

I want to train a model which can predict whether the output for a given sample is positive or negative. I have tried a couple of classification and regression methods, but the result is not satisfactory. Any suggestion to handle this problem is appreciated.

More information: The values that are near -1 or 1 are more important than the values that are near zero. So, converting the target to {-1,1} is not a good idea. On the other hand, since the majority of the target values are around zero, treating this problem as a regression problem does not obtain a good solution, since it tries to reduce the errors related to the targets with large (absolute) values, so it can not find the true boundary for the small values. I tried a hierarchical method, i.e., first deciding about whether the target is small or large and then decide whether it is positive or negative. But, the results are not satisfactory.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
0
$\begingroup$

Your data doesn't look like it is setup for classification. If you're trying to predict whether the target is negative or positive you should convert the target data to be -1 if (target < 0) and 1 if (target >= 0) and use a tanh activation function on your output layer. You could also convert to 0 and 1 and use the sigmoid output layer.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Thanks. It is obvious. Indeed, I seek more advanced suggestions. $\endgroup$
    – Hossein
    Commented Oct 31, 2018 at 20:40
  • $\begingroup$ Can you provide more details on what you're trying to do? I'm not sure what you're trying to do that converting the targets in your histogram to 0 or 1 and performing logistic regression doesn't already do, and more efficiently. $\endgroup$
    – Chad Sexington
    Commented Oct 31, 2018 at 22:13
  • $\begingroup$ My problem is that since the training targets are concentrated around zero, converting it to 0/1 does not get reasonable results. $\endgroup$
    – Hossein
    Commented Nov 11, 2018 at 14:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.