Is this really how p-values work? Can a million research papers per year be based on pure randomness? I'm very new to statistics, and I'm just learning to understand the basics, including $p$-values. But there is a huge question mark in my mind right now, and I kind of hope my understanding is wrong. Here's my thought process:
Aren't all researches around the world somewhat like the monkeys in the "infinite monkey theorem"? Consider that there are 23887 universities in the world. If each university has 1000 students, that's 23 million students each year.
Let's say that each year, each student does at least one piece of research, using hypothesis testing with $\alpha=0.05$.
Doesn't that mean that even if all the research samples were pulled from a random population, about 5% of them would "reject the null hypothesis as invalid". Wow. Think about that. That's about a million research papers per year getting published due to "significant" results.
If this is how it works, this is scary. It means that a lot of the "scientific truth" we take for granted is based on pure randomness.
A simple chunk of R code seems to support my understanding:
library(data.table)
dt <- data.table(p=sapply(1:100000,function(x) t.test(rnorm(10,0,1))$p.value))
dt[p<0.05,]

So does this article on successful $p$-fishing: I Fooled Millions Into Thinking Chocolate Helps Weight Loss. Here's How.
Is this really all there is to it? Is this how "science" is supposed to work?
 A: This is certainly a valid concern, but this isn't quite right. 
If 1,000,000 studies are done and all the null hypotheses are true then approximately 50,000 will have significant results at p < 0.05.  That's what a p value means.  However, the null is essentially never strictly true. But even if we loosen it to "almost true" or "about right" or some such, that would mean that the 1,000,000 studies would all have to be about things like


*

*The relationship between social security number and IQ

*Is the length of your toes related to the state of your birth?


and so on. Nonsense. 
One trouble is, of course, that we don't know which nulls are true.  Another problem is the one @Glen_b mentioned in his comment - the file drawer problem.
This is why I so much like Robert Abelson's ideas that he puts forth in Statistics as Principled Argument.  That is, statistical evidence should be part of a principled argument as to why something is the case and should be judged on the MAGIC criteria:


*

*Magnitude: How big is the effect?

*Articulation: Is it full of "ifs", "ands" and "buts" (that's bad)

*Generality: How widely does it apply?

*Interestingness

*Credibilty: Incredible claims require a lot of evidence

A: Just to add to the discussion, here is an interesting post and subsequent discussion about how people are commonly misunderstanding p-value.
What should be retained in any case is that a p-value is just a measure of the strength of evidence in rejecting a given hypothesis. A p-value is definitely not a hard threshold below which something is "true" and above which it is only due to chance. As explained in the post referenced above:

results are a combination of real effects and chance, it’s not
  either/or

A: 
Aren't all researches around the world somewhat like the "infinite
  monkey theorem" monkeys? 

Remember, scientists are critically NOT like infinite monkeys, because their research behavior--particularly experimentation--is anything but random. Experiments are (at least supposed to be) incredibly carefully controlled manipulations and measurements that are based on mechanistically informed hypotheses that builds on a large body of previous research. They are not just random shots in the dark (or monkey fingers on typewriters).

Consider that there are 23887 universities in
  the world. If each university has 1000 students, that's 23 millions of
  students each year. Let's say that each year, each student does at
  least one research,

That estimate for the number of published research findings has got to be way way off. I don't know if there are 23 million "university students" (does that just include universities, or colleges too?) in the world, but I know that the vast majority of them never publishes any scientific findings. I mean, most of them are not science majors, and even most science majors never publish findings.
A more likely estimate (some discussion) for number of scientific publications each year is about 1-2 million. 

Doesn't that mean that even if all the research samples were pulled
  from random population, about 5% of them would "reject the null
  hypothesis as invalid". Wow. Think of that. That's about a million
  research papers per year getting published due to "significant"
  results.

Keep in mind, not all published research has statistics where significance is right at the p = 0.05 value. Often one sees p values like p<0.01 or even p<0.001. I don't know what the "mean" p value is over a million papers, of course.

If this is how it works, this is scary. It means that a lot of the "scientific truth" we take for granted is based on pure randomness. 

Also keep in mind, scientists are really not supposed to take a small number of results at p around 0.05 as "scientific truth". Not even close. Scientists are supposed to integrate over many studies, each of which has appropriate statistical power, plausible mechanism, reproducibility, magnitude of effect, etc., and incorporate that into a tentative model of how some phenomenon works. 
But, does this mean that almost all of science is correct? No way. Scientists are human, and fall prey to biases, bad research methodology (including improper statistical approaches), fraud, simple human error, and bad luck. Probably more dominant in why a healthy portion of published science is wrong are these factors rather than the p<0.05 convention. In fact, let's just cut right to the chase, and make an even "scarier" statement than what you have put forth:
Why Most Published Research Findings Are False
A: As also pointed out in the other answers, this will only cause problems if you are going to selectively consider the positive results where the null hypothesis is ruled out. This is why scientists write review articles where they consider previously published research results and try to develop a better understanding of the subject based on that. However, there then still remains a problem, which is due to the so-called "publication bias", i.e. scientists are more likely to write up an article about a positive result than on a negative result, also a paper on a negative result is more likely to get rejected for publication than a paper on a positive result.
Especially in fields where statistical test are very important will this be a big problem, the field of medicine is a notorious example. This is why it was made compulsory to register clinical trials before they are conducted (e.g. here). So, you must explain the set up, how the statistical analysis is going to be performed, etc. etc. before the trial gets underway. The leading medical journals will refuse to publish papers if the trials they report on where not registered.
Unfortunately, despite this measure, the system isn't working all that well.
A: This is close to a very important fact about the scientific method: it emphasizes falsifiability.  The philosophy of science which is most popular today has Karl Popper's concept of falsifiability as a corner stone.
The basic scientific process is thus:


*

*Anyone can claim any theory they want, at any time.  Science will admit any theory which is "falsifiable."  The most literal sense of that word is that, if anyone else doesn't like the claim, that person is free to spend the resources to disprove the claim.  If you don't think argyle socks cure cancer, you are free to use your own medical ward to disprove it.

*Because this bar for entry is monumentally low, it is traditional that "Science" as a cultural group will not really entertain any idea until you have done a "good effort" to falsify your own theory.

*Acceptance of ideas tends to go in stages.  You can get your concept into a journal article with one study and a rather low p-value.  What that does buy you is publicity and some credibility.  If someone is interested in your idea, such as if your science has engineering applications, they may want to use it.  At that time, they are more likely to fund an additional round of falsification.

*This process goes forward, always with the same attitude: believe what you want, but to call it science, I need to be able to disprove it later.
This low bar for entry is what allows it to be so innovative.  So yes, there are a large number of theoretically "wrong" journal articles out there.  However, the key is that every published article is in theory falsifiable, so at any point in time, someone could spend the money to test it.
This is the key: journals contain not only things which pass a reasonable p-test, but they also contain the keys for others to dismantle it if the results turn out to be false.
A: Your understanding of $p$-values seems to be correct.
Similar concerns are voiced quite often. What makes sense to compute in your example, is not only the number of studies out of 23 mln that arrive to false positives, but also the proportion of studies that obtained significant effect that were false. This is called "false discovery rate". It is not equal to $\alpha$ and depends on various other things such as e.g. the proportion of nulls across your 23 mln studies. This is of course impossible to know, but one can make guesses. Some people say that the false discovery rate is at least 30%.
See e.g. this recent discussion of a 2014 paper by David Colquhoun: Confusion with false discovery rate and multiple testing (on Colquhoun 2014). I have been arguing there against this "at least 30%" estimate, but I do agree that in some fields of research false discovery rate can be a lot bit higher than 5%. This is indeed worrisome.
I don't think that saying that null is almost never true helps here; Type S and Type M errors (as introduced by Andrew Gelman) are not much better than type I/II errors.
I think what it really means, is that one should never trust an isolated "significant" result.
This is even true in high energy physics with their super-stringent $\alpha\approx 10^{-7}$ criterion; we believe the discovery of the Higgs boson partially because it fits so well to the theory prediction. This is of course much much MUCH more so in some other disciplines with much lower conventional significance criteria ($\alpha=0.05$) and lack of very specific theoretical predictions.
Good studies, at least in my field, do not report an isolated $p<0.05$ result. Such a finding would need to be confirmed by another (at least partially independent) analysis, and by a couple of other independent experiments. If I look at the best studies in my field, I always see a whole bunch of experiments that together point at a particular result; their "cumulative" $p$-value (that is never explicitly computed) is very low.
To put it differently, I think that if a researcher gets some $p<0.05$ finding, it only means that he or she should go and investigate it further.  It definitely does not mean that it should be regarded as "scientific truth".
A: A substantial check on the important issue raised in this question is that "scientific truth" is not based on individual, isolated publications. If a result is sufficiently interesting it will prompt other scientists to pursue the implications of the result. That work will tend to confirm or refute the original finding. There might be a 1/20 chance of rejecting a true null hypothesis in an individual study, but only a 1/400 of doing so twice in a row.
If scientists did simply repeat experiments until they find "significance" and then published their results the problem might be as large as the OP suggests. But that's not how science works, at least in my nearly 50 years of experience in biomedical research. Furthermore, a publication is seldom about a single "significant" experiment but rather is based on a set of inter-related experiments (each required to be "significant" on its own) that together provide support for a broader, substantive hypothesis.
A much larger problem comes from scientists who are too committed to their own hypotheses. They then may over-interpret the implications of individual experiments to support their hypotheses, engage in dubious data editing (like arbitrarily removing outliers), or (as I have seen and helped catch) just make up the data.
Science, however, is a highly social process, regardless of the mythology about mad scientists hiding high up in ivory towers. The give and take among thousands of scientists pursuing their interests, based on what they have learned from others' work, is the ultimate institutional protection from false positives. False findings can sometimes be perpetuated for years, but if an issue is sufficiently important the process will eventually identify the erroneous conclusions.
A: 
Is this how "science" is supposed to work?

That's how a lot of social sciences work. No so much with physical sciences. Think of this: you typed your question on a computer. People were able to build these complicated beasts called computers using the knowledge of physics, chemistry and other fields of physical sciences. If the situation was as bad as you describe, none of the electronics would work. Or think of the things like a mass of an electron, which is known with insane precision. They pass through billions of logic gates in a computer over an over, and your computer still works and works for years.
UPDATE:
To respond to the down votes I received, I felt inspired to give you a couple of examples.
The first one is from physics: Bystritsky, V. M., et al. "Measuring the astrophysical S factors and the cross sections of the p (d, γ) 3He reaction in the ultralow energy region using a zirconium deuteride target." Physics of Particles and Nuclei Letters 10.7 (2013): 717-722.
As I wrote before, these physicist don't even pretend doing any statistics beyond computing the standard errors. There's a bunch of graphs and tables, not a single p-value or even confidence interval. The only evidence of statistics is the standard errors notes as $0.237 \pm 0.061$, for instance. 
My next example is from... psychology: Paustian-Underdahl, Samantha C., Lisa Slattery Walker, and David J. Woehr. "Gender and perceptions of leadership effectiveness: A meta-analysis of contextual moderators." Journal of Applied Psychology, 2014, Vol. 99, No. 6, 1129 –1145. 
These researchers have all the usual suspects: confidence intervals, p-values, $\chi^2$ etc.
Now, look at some tables from papers and guess which papers they are from:


That's the answer why in one case you need "cool" statistics and in another you don't: because the data is either crappy or not. When you have good data, you don't need much stats beyond standard errors.
UPDATE2:
@PatrickS.Forscher made an interesting statement in the comment: 

It is also true that social science theories are "softer" (less
  formal) than physics theories.

I must disagree. In Economics and Finance the theories are not "soft" at all. You can randomly lookup a paper in these fields and get something like this:

and so on.
It's from Schervish, Mark J., Teddy Seidenfeld, and Joseph B. Kadane. "Extensions of expected utility theory and some limitations of pairwise comparisons." (2003). Does this look soft to you?
I'm re-iterating my point here that when your theories are not good and the data is crappy, you can use the hardest math and still get a crappy result.
In this paper they're talking about utilities, the concept like happiness and satisfaction - absolutely unobservable. It's like what is a utility of having a house vs. eating a cheeseburger? Presumably there's this function, where you can plug "eat cheeseburger" or "live in own house" and the function will spit out the answer in some units. As crazy as it sounds this is what modern ecnomics is built on, thank to von Neuman.
