I'm going to approach this from the alternate direction of philosophy, in light of the really useful principles of Uncertainty Management discussed in George F. Klir's books on fuzzy sets. I can't give van der Laan exactness, but I can provide a somewhat exhaustive case for why his goal is logically impossible; that is going to call for a lengthy discussion that references other fields, so bear with me.
Klir and his co-authors divide uncertainty into several subtypes, such as nonspecificity (i.e. when you have an unknown set of alternatives, dealt with through means like the Hartley Function); imprecision in definitions (i.e. the "fuzziness" modeled and quantified in fuzzy sets); strife or discord in evidence (addressed in Dempster-Shafer Evidence Theory); plus probability theory, possibility theory and measurement uncertainty, where the goal is to have an adequate scope to capture the relevant evidence, while minimizing errors. I look at the whole toolbox of statistical techniques as alternate means of partitioning uncertainty in different ways, much like a cookie cutter; confidence intervals and p-values quarantine uncertainty in one way, while measures like Shannon's Entropy whittle it down from another angle. What they can't do, however, is eliminate it entirely. To achieve an "exact model" of the kind van der Laan seems to describe, we'd need to reduce all of these types of uncertainty down to zero, so that there's no more left to partition. A truly "exact" model would always have probability and possibility values of 1, nonspecificity scores of 0 and no uncertainty whatsoever in the definitions of terms, ranges of values or measurement scales. There would be no discord in alternate sources of evidence. The predictions made by such a model would always be 100 percent accurate; predictive models essentially partition their uncertainty off into the future, but there would be none left to put off. The uncertainty perspective has some important implications:
• This tall order is not only physically implausible, but actually logically impossible. Obviously, we cannot achieve perfectly continuous measurement scales with infinitesimal degrees, by gathering finite observations using fallible, physical scientific equipment; there will always be some uncertainty in terms of measurement scale. Likewise, there will always be some fuzziness surrounding the very definitions we employ in our experiments. The future is also inherently uncertain, so the supposedly perfect predictions of our "exact" models will have to be treated as imperfect until proven otherwise - which would take an eternity.
• To make matters worse, no measurement technique is 100 percent free of error at some point in the process, nor can it be made comprehensive enough to embrace all of the possibly conflicting information in the universe.
Furthermore, the elimination of possible confounding variables and complete conditional independence cannot be proven thoroughly without examining all other physical processes that affect the one we're examining, as well as those that affect these secondary processes and so on.
• Exactness is possible only in pure logic and its subset, mathematics, precisely because abstractions are divorced from real-world concerns like these sources of uncertainty. For example, by pure deductive logic, we can prove that 2 + 2 = 4 and any other answer is 100 percent incorrect. We can also make perfectly accurate predictions that it will always equal 4. This kind of precision is only possible in statistics when we're dealing with abstractions. Statistics is incredibly useful when applied to the real world, but the very thing that makes it useful injects at least some degree of inescapable uncertainty, thereby rendering it inexact. It is an unavoidable dilemma.
• Furthermore, Peter Chu raises additional limitations in the comments section of the article rvl linked to. He puts it better than I can:
"This solution surface of NP-hard problems is typically rife with many
local optima and in most case it is computationally unfeasible to
solve the problem i.e. finding the global optimal solution in general.
Hence, each modeler is using some (heuristic) modeling techniques, at
best, to find adequate local optimal solutions in the vast solution
space of this complex objective function."
• All of this means that science itself cannot be perfectly accurate, although van der Laan seems to speak of it in this way in his article; the scientific method as an abstract process is precisely definable, but the impossibility of universal and perfect exact measurement means it cannot produce exact models devoid of uncertainty. Science is a great tool, but it has limits.
• It gets worse from there: Even if were possible to exactly measure all of the forces acting on every constituent quark and gluon in the universe, some uncertainties would still remain. First, any predictions made by such a complete model would still be uncertain due to the existence of multiple solutions for quintic equations and higher polynomials. Secondly, we cannot be completely certain that the extreme skepticism in embodied in the classic question "maybe this is all a dream or a hallucination" is not a reflection of reality - in which case all of our models are indeed wrong in the worst possible way. This is basically equivalent to a more extreme ontological interpretation of the original epistemological formulations of philosophies like phenomenalism, idealism and solipsism.
• In his 1909 classic Orthodoxy G.K. Chesterton noted that the extreme versions of these philosophies can indeed be judged, but by whether or not they drive their believers into mental institutions; ontological solipsism, for example, is actually a marker of schizophrenia, as are some of its cousins. The best that we can achieve in this world is to eliminate reasonable doubt; unreasonable doubt of this unsettling kind cannot be rigorously done away with, even in a hypothetical world of exact models, exhaustive and error-free measurements. If van der Laan aims at ridding us of unreasonable doubt then he is playing with fire. By grasping at perfection, the finite good we can do will slip through our fingers; we are finite creatures existing in an infinite world, which means the kind of complete and utterly certain knowledge van der Laan argues for is permanently beyond our grasp. The only way we can reach that kind of certainty is by retreating from that world into the narrower confines of the perfectly abstract one we call "pure mathematics." This does not mean, however, that a retreat into pure mathematics is the solution to eliminating uncertainty. This was essentially the approach taken by the successors of Ludwig Wittgenstein (1889-1951), who drained his philosophy of logical positivism of whatever common sense it had by rejecting metaphysics altogether and retreating entirely into pure math and scientism, as well as extreme skepticism, overspecialization and overemphasis on exactness over usefulness. In the process, they destroyed the discipline of philosophy by dissolving it into a morass of nitpicking over definitions and navel-gazing, thereby making it irrelevant to the rest of academia. This essentially killed the whole discipline, which had still been at the forefront of academic debate until the early 20th Century, to the point where it still garnered media attention and some of its leaders were household names. They grasped at a perfect, polished explanation of the world and it slipped through their fingers - just as it did through the mental patients GKC spoke of. It will also slip out of the grasp of van der Laan, who has already disproved his own point, as discussed below. The pursuit of models that are too exact is not just impossible; it can be dangerous, if taken to the point of self-defeating obsession. The pursuit of that kind of purity rarely ends well; it's often as self-defeating as those germophobes who scrub their hands so furiously that they end up with wounds that get infected. It's reminiscent of Icarus trying to steal fire from the Sun: as finite beings, we can have only a finite understanding of things. As Chesterton also says in Orthodoxy, "It is the logician who
seeks to get the heavens into his head. And it is his head that splits."
In the light of the above, let me tackle some of the specific questions listed by rvl:
1) A model with no assumptions whatsoever is either a) not aware of its own assumptions or b) must be cleanly divorced from considerations that introduce uncertainty, such as measurement errors, accounting for every single possible confounding variable, perfectly continuous measurement scales and the like.
2) I'm still a newbie when it comes to maximum likelihood estimation (MLE), so I can't comment on the mechanics of target likelihood, except to point out the obvious: likelihood is just that, a likelihood, not a certainty. To derive an exact model requires complete elimination of uncertainty, which probabilistic logic can rarely do, if ever.
3) Of course not. Since all models retain some uncertainty and are thus inexact (except in cases of pure mathematics, divorced from real-world physical measurements), the human race would not have been able to make any technological progress to date - or indeed, any other progress at all. If inexact models were always useless, we'd be having this conversation in a cave, instead of on this incredible feat of technology called the Internet, all of which was made possible through inexact modeling.
Ironically, van der Laan's own model is a primary example of inexactness. His own article sketches out a model of sorts of how the field of statistics ought to be managed, with an aim towards exact models; there are no numbers attached to this "model" yet, no measurement of just how inexact or useless most models are now in his view, no quantification of how far we are away from his vision, but I suppose one could devise tests for those things. As it stands, however, his model is inexact. If it is not useful, it means his point is wrong; if it is useful, it defeats his main point that inexact models aren't useful. Either way, he disproves his own argument.
4) Probably not, because we cannot have complete information to test our model with, for the same reasons that we can't derive an exact model in the first place. An exact model would by definition require perfect predictability, but even if the first 100 tests turn out 100 percent accurate, the 101st might not. Then there's the whole issue of infinitesimal measurement scales. After that, we get into all of the other sources of uncertainty, which will contaminate any Ivory Tower evaluation of our Ivory Tower model.
5) To address the issue, I had to put it in the wider context of much larger philosophical issues that are often controversial, so I don't think it's possible discuss this without getting into opinions (note how that in and of itself is another source of uncertainty) but you're right, this article deserves a reply. A lot of what he says on other topics is on the right track, such as the need to make statistics relevant to Big Data, but there is some impractical extremism mixed in there that should be corrected.
