There are four sources of differences between Bayesian, Likelihoodist, Frequentist, and the various uncategorized miscellaneous methods, such as the method of moments, that I have been able to discover. The first has to with differences in defining the idea of an optimal solution. The second has to do with the deeper layers of the methods, such as the axiom structure. The third has to do with prior knowledge. The fourth has to do with the intent of the model.
My daughter and I were home some very many years ago, and neither my wife nor sons were home. My guess is that they were involved in a band activity or maybe lacrosse. We had cake the day before, and when I opened the refrigerator, I realized that there was too much cake for one person but not enough cake for two people. I asked her if she wanted to split the last piece. I grabbed a knife to cut it, and she said, "can I cut the cake?"
I said, "sure, but if you cut, then I get to choose the piece." That seemed most amenable to her little girl's mind, and I could see the wheels turning as she moved the knife around to work out how uneven she should cut the cake and not raise any protest. She finally made her cut, and I grabbed my piece of cake.
A loud protest rang out, and I offered to put the pieces back together again to try and make the cut better. Her bewildered look was followed by an emphatic "no," from her. So I shrugged, walked off, and ate the cake leaving her with the small piece.
It was then that she realized her father was related to Darth Vader.
My daughter had used her prior knowledge of when I cut the cake and let her pick, had assigned a zero prior probability to my statement, and chose her fair cut. The likelihood, when combined with her posterior, changed her view of many parameters.
This mentality of "I cut the cake, and you choose" is the basis of one of the axiomatizations of probability. I have just shown you its potential weakness. You should always assume a being of utter darkness is on the other side of the deal. My daughter now understands why I cast a shadow even when there is no direct source of light.
Let us assume that I will let you set the gambling odds, but I will choose what combination of bets to take. Presumably, you will not accept a gamble where I am certain to win regardless of the outcome of the event being gambled on. You won't play, "heads, I win; tails, you lose."
So we are going to play a cake-cutting game and then gamble on it. By gambling on it, we change the intention of the game. We also invoke a very particular axiom set that we can contrast with another set. In addition, we can ponder the idea of optimality. Tangentially, we can discuss the small impact prior knowledge has.
By making you the financial intermediary, I am giving you control of the "bid" and the "ask," or alternatively, the vig, using bookies' terminology. To make the game fair, we agree that I will pay a flat fee per cut cake. In return, I can bet any finite sum. I can either go long or short on any gamble as well. We will assume that I have adequate collateral to cover any offered short bet.
Also, to make the game fair, you agree to use the risk-neutral measure as you are collecting a flat fee. In other words, you will grant fair odds. The reason for the flat fee is that lump sums are constants, and so their derivative is zero. The fee does not play a factor in profit optimization.
You have two tasks. The first is to cut the cake, the second to give prices for lotteries. Either the left side or the right side of the cake will be larger.
In the simple games of economics without uncertainty, you would want the cake to be cut as follows in the illustration. You want it to be cut in such a way that you cannot distinguish which side is bigger.
Unfortunately, this is not a game of perfect knowledge. We need to play a statistical game that has uncertainty in it, or it isn't any fun.
To add uncertainty, we will have a robot put the cake in a darkened room so that neither can see the cake's location. A point on the table will be chosen by a random number generator. The generator will produce uniformly distributed points.
A robot will enter the room and place the center of this perfectly circular cake on the randomly chosen point. A sensor will detect points from the cake in a uniformly distributed manner. In other words, both you and I will be provided with forty sets of points. Each point on the cake is equiprobable to be detected by the sensor.
You will then cut the cake through the minimum variance unbiased estimator (MVUE) of the center using an ultrapowerful laser cutter.
Because the MVUE is guaranteed by force of math to be perfectly accurate, half the time the left side will be larger, as the sample size tends to infinity. If you granted even odds, and I always bet on the left side, one would expect a binomial distribution of outcomes over thirty cakes to look thusly.
But would betting on left every time be a rational decision on my part?
After all, I have data. I could form a posterior distribution for the parameter and do another estimate. Since I have a computer, I could do it before you cut the cake, based only on our shared forty points. What if my Bayesian estimate didn't agree with your Frequentist estimate?
Now, this is where a bunch of mathematical tricks comes in. The intent is to gamble, which changes the outcome. If we were not gambling, although there would be differences, they wouldn't really matter. Indeed, all that would matter would be how we defined optimality upfront, determining which solution we should choose.
Gambling is different because Frequentist statistics give rise to arbitrage opportunities at least in some percentage of the repetitions of a game.
The issue has to do with a conflict between the Dutch Book Theorem and Kolmogorov's axioms. The appearance of this conflict varies from game to game but is always present. It sometimes takes a bit of digging to find the conflict, but it will be there as a pathology. There also has to be an opponent. Someone has to know that there is a mispricing of a lottery such as a stock, horse racing ticket, or options on wheat futures.
The specific reason is that Kolmogorov's third axiom is that the union of partitions of sets yield countably additive measures. You can cut a continuous probability distribution into a countably infinite number of parts under Frequentist axioms. The Dutch Book Theorem says that you cannot do that. You can cut sets, but the number of sets must be finite.
My objective function would be to win the largest number of gambles. It would not care if I used an unbiased method unless that maximizes wins.
Now the prior does give me a slight edge. A cake near the border will, in part, sit off the table. Points over the edge cannot sit in a location that could also be the center of the cake. If all the observed points came from places over the edge, even though that would be an exceedingly rare combination of events, it would allow me to know that you have a mistake in your calculations. If the MVUE were sitting in midair instead of on the table, I would know your calculations were wrong.
Nonetheless, we have forty data points. If we expanded the table enough, we could make that case vanishingly slight.
Still, there is a problem. I have data from which I can construct a posterior density. Bayesian bets cannot be arbitraged under mild additional conditions. Lotteries priced with Frequentist tools can be arbitraged at least some of the time.
The Frequentist cake cut would pass through the circle like this. Please note that the circle is an oval on some machines due to differences in devices and settings.
However, the MVUE is not inside the posterior. You can know that because you can place a circle of equal radius around the MVUE, and you will get the following graph.
Obviously, the MVUE sits in a physically impossible location. That permits anybody to know which side will be larger. Knowing the points is sufficient to detect the differences between the two types of models against the reality.
The green lines map the Bayesian posterior. The black point minimizes quadratic loss, but each point in the posterior is an equally valid possible location for the center of the cake.
I did a simulation putting a corner at (10,10) and (90,90), and I was guaranteed a win 48% of the time. Additionally, I won more than half the remaining gambles due to differences between the methods.
For the MVUE to be guaranteed to be unbiased, it has to pay for that insurance policy with information. That difference in information results in a difference in precision.
Because you are cutting at an angle from (0,0), you must be outside the joint Cartesian posterior and the marginal distribution of the posterior angle or slope. The MVUE can be closer to the true center even though it is outside the Cartesian posterior.
If I used your cuts and your odds for thirty rounds, my expected value, assuming I bet 100% of my money every round I knew the correct answer and made the Kelly-optimal bet the remaining 52% of the time, would be to make 128,000 times my initial amount. Under the Frequentist model where I always bet left or a random location, I would expect a small loss due to the fee I was paying to play.
I have a half dozen of these games related to asset pricing, trading, or options pricing.
Real-world financial models are complicated because if you attempt to derive the Frequentist models in a Bayesian framework, you do not end up with models that look alike at all. It doesn't help, necessarily, to cut and paste a Bayesian procedure on an economic model built on top of Frequentist axioms.
Nonetheless, you can learn quite a bit when you cut a cake with a child involved.