The objective is to classify a variable Y which is binary, outcomes [0,1], where all the features $X_1$....$X_{156}$ are normalized and continuous.

The methods I'm using are Logistic regression and XGBoost, I'm rather new when it come to this method.

The result after using feature selection gives an AUC a bit under $70\%$. The reason I think that the result is rather poor might be because of the data.

So I've got data with the following dimension:

156 features and 625 data points. It would seem that the sample size is too small in relation to the amount of features.

Thus I wonder if a resampling method would be adequate such as bootstrap?

  • $\begingroup$ What's the objective and data type? $\endgroup$ – user2974951 Oct 15 '18 at 11:59
  • 2
    $\begingroup$ Your question is rather unclear. What are you trying to do? My blog post how to ask a statistics question may help you formulate your question in a way that can be answered. $\endgroup$ – Peter Flom - Reinstate Monica Oct 15 '18 at 11:59
  • $\begingroup$ edited the question, added some more details $\endgroup$ – Erik Castillo Oct 15 '18 at 12:53
  • 3
    $\begingroup$ As yourself this question. You go to the store and at the checkout counter the bill comes to \$20. You have $10. Would the clerk be willing to magnify the \$10 to equal \$20? You can't create data, or money. $\endgroup$ – Frank Harrell Oct 15 '18 at 12:58

I think that depends on your objective and data. I will assume that your task is machine learning since you use the “feature” term:

156 features seem to have a high cardinality proportional to 625 samples. If your data is not a time-series sequence, you can try to use bootstrapping or Monte-Carlo simulations; neuroscience researchers sometimes extract data from real human and animals manually, creating those can consume so much time and can be costly so that they sometimes end up with small amount of samples as you. Moreover, if you get computationally stuck, you may look up for jack-knifing your samples. But without knowing the data, using those methods can be dangerous, try to describe your features at least statistically and to plot them to observe what kind of distributions you have. Know your data.

However, if you have a time-series sequence; you may look up for another approach which I do not have the knowledge of. Direct bootstrapping of a time-series sequence may be unsafe, you may need a clever approach.

As last note: In any case, I’d suggest you to still try your dataset for your task. If most of your features are useless or highly correlated with each other; your number of actual useful features can be actually 20, 25 etc. Moreover, the patterns among the features can be so explicit for machine learning algorithms such that 625 samples can be surprisingly enough for modeling or some other analysis. Still, know your data.

  • $\begingroup$ I disagree. If your data is not from time series that does not mean you can simply use bootstrapping. If we do not know more about the data we cannot say whether or not bootstrapping is a good or terrible idea. $\endgroup$ – Jan Oct 15 '18 at 13:04
  • $\begingroup$ Think of bootstrapping a financial time-series signal. If you apply bootstrapping directly, how can it make a reasonable sequence? New samples of stock closing prices could have 2000$ at time t, 4400$ at t-1, 300$ at t-2. Time-series signals are not i.i.d or randomly sampled. Each sample is correlated with its past or future values. But I say that, it may be possible to bootstrap a time-series signal with a clever approach(integration of a ARIMA model maybe), just not the direct way. I do not know it, if you know such a method, please explain, that would be really good to know, sincerely. $\endgroup$ – Ugur MULUK Oct 15 '18 at 14:31
  • $\begingroup$ You took my comment the wrong way. I am simply saying that without more knowledge on the data it is dangerous to assume that we can use bootstrapping. $\endgroup$ – Jan Oct 15 '18 at 14:33
  • 1
    $\begingroup$ Jan, I think I’ve focused on your first sentence. I agree with you about the knowledge on the data, so that I edited my answer accordingly; I made some of my arguments lighter and added some warnings. Regards. $\endgroup$ – Ugur MULUK Oct 15 '18 at 14:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.