Machine learning, overfitting and artificial sample increments (LASSO, Boosting)

Question

I am trying to use different types of Machine Learning (ML), LASSO, Elastic Net and Boosting, in a dataset with around 6000 observations and 120 regressors. To test the goodness-of-fit of results, I used an out-of-sample technique: I create a subset (training set) and availing the result in the remaining sample (test set) to make the prediction. Next, I have compared the result of this prediction with the real value of the dependent variable. I made it for ten times (training-test trial).

I noted important overfitting. In my opinion, this effect is a consequence of biased estimation in training set in turn caused by zero-inflated values in the sample ( only 22% of the total sample have a value not zero).

I thought to increase the number of observations, simply, making a dataset with 30.000 observation, deriving from the five copies of original observations. To justify this approach, in my view, use a weak Law of Large Numbers (wLLT). The result is much encouragement and the problem of overfitting very limited. I accept every suggests or critiques of this approach to implement my study.

EDIT: My original database is stratified.

Answer 1

I thought to increase the number of observations, simply, making a dataset with 30.000 observation, deriving from the five copies of original observations.

Are you saying you simply multiplied the data to increase sample size?

There is a reason this is not advised:

Consider you draw a random sample of the population to estimate the average wealth of the average person. By chance, you happen to sample a doctor, an oil tycoon, a famous model, and a CEO. If you suspect your small sample (4) is biased at predicting average wealth, does multiplying the data help? No. It actually makes it worse.