0
$\begingroup$

The objective is to classify a variable Y which is binary, outcomes [0,1], where all the features $X_1$....$X_{156}$ are normalized and continuous.

The methods I'm using are Logistic regression and XGBoost, I'm rather new when it come to this method.

The result after using feature selection gives an AUC a bit under $70\%$. The reason I think that the result is rather poor might be because of the data.

So I've got data with the following dimension:

156 features and 625 data points. It would seem that the sample size is too small in relation to the amount of features.

Thus I wonder if a resampling method would be adequate such as bootstrap?

$\endgroup$
4
  • $\begingroup$ What's the objective and data type? $\endgroup$ Oct 15, 2018 at 11:59
  • 2
    $\begingroup$ Your question is rather unclear. What are you trying to do? My blog post how to ask a statistics question may help you formulate your question in a way that can be answered. $\endgroup$
    – Peter Flom
    Oct 15, 2018 at 11:59
  • $\begingroup$ edited the question, added some more details $\endgroup$ Oct 15, 2018 at 12:53
  • 3
    $\begingroup$ As yourself this question. You go to the store and at the checkout counter the bill comes to \$20. You have $10. Would the clerk be willing to magnify the \$10 to equal \$20? You can't create data, or money. $\endgroup$ Oct 15, 2018 at 12:58

1 Answer 1

-1
$\begingroup$

I think that depends on your objective and data. I will assume that your task is machine learning since you use the “feature” term:

156 features seem to have a high cardinality proportional to 625 samples. If your data is not a time-series sequence, you can try to use bootstrapping or Monte-Carlo simulations; neuroscience researchers sometimes extract data from real human and animals manually, creating those can consume so much time and can be costly so that they sometimes end up with small amount of samples as you. Moreover, if you get computationally stuck, you may look up for jack-knifing your samples. But without knowing the data, using those methods can be dangerous, try to describe your features at least statistically and to plot them to observe what kind of distributions you have. Know your data.

However, if you have a time-series sequence; you may look up for another approach which I do not have the knowledge of. Direct bootstrapping of a time-series sequence may be unsafe, you may need a clever approach.

As last note: In any case, I’d suggest you to still try your dataset for your task. If most of your features are useless or highly correlated with each other; your number of actual useful features can be actually 20, 25 etc. Moreover, the patterns among the features can be so explicit for machine learning algorithms such that 625 samples can be surprisingly enough for modeling or some other analysis. Still, know your data.

$\endgroup$
4
  • $\begingroup$ I disagree. If your data is not from time series that does not mean you can simply use bootstrapping. If we do not know more about the data we cannot say whether or not bootstrapping is a good or terrible idea. $\endgroup$
    – Jan
    Oct 15, 2018 at 13:04
  • $\begingroup$ Think of bootstrapping a financial time-series signal. If you apply bootstrapping directly, how can it make a reasonable sequence? New samples of stock closing prices could have 2000$ at time t, 4400$ at t-1, 300$ at t-2. Time-series signals are not i.i.d or randomly sampled. Each sample is correlated with its past or future values. But I say that, it may be possible to bootstrap a time-series signal with a clever approach(integration of a ARIMA model maybe), just not the direct way. I do not know it, if you know such a method, please explain, that would be really good to know, sincerely. $\endgroup$
    – Ugur MULUK
    Oct 15, 2018 at 14:31
  • $\begingroup$ You took my comment the wrong way. I am simply saying that without more knowledge on the data it is dangerous to assume that we can use bootstrapping. $\endgroup$
    – Jan
    Oct 15, 2018 at 14:33
  • 1
    $\begingroup$ Jan, I think I’ve focused on your first sentence. I agree with you about the knowledge on the data, so that I edited my answer accordingly; I made some of my arguments lighter and added some warnings. Regards. $\endgroup$
    – Ugur MULUK
    Oct 15, 2018 at 14:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.