Are most published correlations in social sciences untrustworthy and what is to be done about it? Despite the important but smacking of "gotcha"-istic efforts by individuals to reveal the practices of predatory journals, a greater and more fundamental threat looms in the shadows of social science research (though there are certainly multiple problems that researchers need to address). To get straight to the point, according to one view we may not be able to trust correlation coefficients derived from samples smaller than 250. 
One would be hard-pressed to find a test more relied upon to infer the presence, direction, and strength of association between to measures in social science than the trusted correlation coefficient. However, one would not be hard pressed to find peer-reviewed reports making strong claims about the relation between two constructs based on correlation coefficients calculated from data with fewer than 250 cases. 
Given the current replication crisis facing social sciences (see the second link above), how should we view this report regarding the stabilization of correlation coefficients only at large samples (at least by some social science field standards)? Is it another crack in the wall of peer-reviewed social science research, or is it a relatively trivial matter that has been overblown in its presentation? 
As there is not likely a single correct answer to this question I hope instead to generate a thread where resources about this question can be shared, thoughtfully considered, and debated (politely and respectfully of course). 
 A: Adding confidence intervals for the estimated true correlation coefficients $\rho$ would be a small (and very simple) first step in the right direction. Its width immediately gives you an impression on the precision of your sample correlation and, at the same time, allows the writer and also the audience to test useful hypotheses. What puzzled me always when talking to statisticians from social science that an absolute sample correlation coefficient above $L = 0.3$ (or some other limit) was considered to be meaningful. At the same time, they were testing the working hypothesis $\rho \ne 0$. This is inconsequencial. Why would a very small population correlation coefficient suddenly be considered as being meaningful? The "correct" working hypothesis would be $|\rho| > L$. Having a confidence interval for $\rho$ at hand, hypotheses like this can easily be tested: just check that the interval is located entirely above $L$ (or below $-L$) and you know whether you can claim a "substantial" statistical association even in the population.
Of course just adding a confidence interval and using meaningful tests won't solve too many problems (like bad sampling designs, omitted consideration of confounders etc.). But it is basically for free. I'd guess even SPSS is able to calculate them!
A: As Michael M notes, the trustworthiness of reported correlations - or of any other estimate - can be assessed using confidence intervals. To a degree, that is. CIs will be too narrow if models were selected after data collection, which I estimate to happen about 95% of the time in the social sciences (which I'll honestly state is a complete guess of mine).
The remedy is twofold:


*

*We are talking about a "replicability crisis". Thus, failed replications inform us that the original effect was probably just random noise. We need to do (and fund, and write up, and submit, and accept) more replications. Replication studies are slowly gaining respectability, and that is a good thing.

*The second remedy is of course meta-analysis. If we have many reported correlations of similar data, even if every single one of them has low $n$, then we can pool the information and learn something. Ideally, we will even be able to detect publication-bias in the process.
