I totally agree with Deer Hunter that we don't have enough information to answer your question. Moreover, an answer will usually not be possible from within the data, it will depend on the application and data-generating processes.
For predictive models, ultimately the validation with independent representative test cases will show whether what you did was a good idea or not.
The fact that you have clusters means that you have to be careful if using resampling validation schemes (cross validation or out-of-bootstrap). However, the examples below show that the apparent clusters may not coincide with the important structures in the data generating process. So knowledge of clusters only tells you that you'll probably run into trouble with "naive" resampling of the cases, but you cannot conclude about how the appropriate regression and resampling set-ups. Therefore, an inner loop of resampling validation may not be able to guide your modeling.
Some situations from chemical calibration with different clustering as "symptom" and conclusions for the appropriate regression model differing independent of the type of observed clustering.
In all examples, the concentration of some analyte is to be modeled.
A number of calibration samples is obtained. From each of these calibration samples, a number of measurements is taken.
The reference value may be pre-specified (calibration samples are mixed accordingly), or samples are taken e.g. from a production process and reference values are measured with some other method.
You expect a strong clustering because of the repeated measurements, but per-cluster-regression is utterly meaningless. However, this situation is unproblematic, as you'd immediate notice that the information for the dependent variable is between the clusters, not within.
Calibration samples are often prepared by dilution of a stock solution (German Wiki on serial dilution is much better than the English. Dilution series of several independently prepared stock solutions are measured.
Per-stock-regression may lead to better prediction of measurements from the same stock.
However, this is usually not appropriate, as rarely samples from the same stock are to be predicted.
You may see clusters in the independent data like in situation 1 that come from the fact that concentration series are usually prepared at "pretty" levels.
You may see the different stocks as clusters in the residuals (if instrument noise and random error in arithmetic dilution is << random error in concentration of stock solutions -- usually a bad sign wrt. the preparation).
Situation 3: You run into matrix effects and your data comprises a number of different matrices. In this case, per-matrix-regression is a valid option an can be much more successful than an overall regression.
Like in situation 2, you may encounter systematic patterns in the residuals of an overall regression.
Actually, you'd need to state for each regression model exactly in which situation it is appropriate. Your models cannot be expected to generalize to unknown matrices.