I'm new to machine learning, and want to train a regression model based on 1000000 labelled samples (with, say, r features and 1 target).
On a histogram, I see that the targets approximately follow a normal distribution around 0, with standard deviation of about 0.75.
When I train the model, it tends to spit out predictions that are very close to zero. Specifically, almost all of the predictions are in the order of 0.01 (for example, I mostly see predictions like 0.0205 or -0.005).
What I suspect is going on: since so many targets are close to 0 (the mean is 0 and the standard deviation is small), it's a bit like in classification when we have an imbalanced dataset (too many instances of one class), so the model tends to just spit out predictions falling into in that class. Likewise, here, a lot of the targets are close to 0, so the model is outputting predictions near to 0 a lot of the time.
I don't think the problem has to do with scaling the features (I've tried that but the problem persists). I've tried different models (namely LSTM, linear regression, neural networks) and encounter the same problem.
Is my suspicion correct that this is an example of data imbalance, and if so, what can I do to overcome the problem?