3 added 12 characters in body
source | link

Which is a work of 50 pages and difficult to summarize. But it deals with your questions on unbiasednessunbiasedness, explains the method of least squaresleast squares (and difference with method of maximum likelihood), and specifically provides a treatment of confidence intervalstreatment of confidence intervals (frequentist interval are already not similar, unique, let alone that the are the same as Bayesian intervals for flat priors).

Which is a work of 50 pages and difficult to summarize. But it deals with your questions on unbiasedness, explains the method of least squares (and difference with method of maximum likelihood), and specifically provides a treatment of confidence intervals (frequentist interval are already not similar, unique, let alone that the are the same as Bayesian intervals for flat priors).

Which is a work of 50 pages and difficult to summarize. But it deals with your questions on unbiasedness, explains the method of least squares (and difference with method of maximum likelihood), and specifically provides a treatment of confidence intervals (frequentist interval are already not similar, unique, let alone that the are the same as Bayesian intervals for flat priors).

2 added 728 characters in body
source | link

You make a notion of the Likelihoodlikelihood as being a Bayesian expression with a flat prior, however while the mathematics coincide (when wrongly interpreted, since you may get P(x|a) = P(a|x), but they are not the same terms) the construction and meaning is different.

Likelihood is not meant to be a 'Bayesian probability based on flat, or uniformedHowever, priors'. Likelihood is not even a probability and does not follow the rules of a probability (for instance you can not add up likelihood for different events, and the sum is not equal to one), it is only when you multiply it with a flat prior, that it becomes a probability, but then the meaning has changed as well.

  1. while the mathematics coincide (when wrongly interpreted, since you may get P(x|a) = P(a|x), up to a constant, but they are not the same terms) the construction and meaning is different.

  2. Likelihood is not meant to be a 'Bayesian probability based on flat, or uniformed, priors'. Likelihood is not even a probability and does not follow the rules of probability distributions (for instance you can not add up likelihood for different events, and the integral is not equal to one), it is only when you multiply it with a flat prior, that it becomes a probability, but then the meaning has changed as well.

Bayesian and frequentistsfrequentist methods are different tools:

By constructing a table of corresponding values, we may know as soon as T is calculated what is the fiducial 5 per cent, value of $\theta$, and that the true value of $theta$$\theta$ will be less than this value in just 5 per cent, of trials. This then is a definite probability statement about the unknown parameter $\theta$, which is true irrespective of any assumption as to its a priori distribution.

a frequentist method makes a statement about the probability that an experiment will have the true value of a parameter (which may differ from experiment to experiment) inside the interval given by a statistic. This is not the be confused with the probability that a specific experiment will have the true value of the parameter inside the interval given by the statistic.

  • a frequentist method makes a statement about the probability that an experiment (with random interval) will have the true value of a (possibly random) parameter inside the interval given by a statistic.
  • This is not the be confused with the probability that a specific experiment (with fixed interval) will have the true value of the (fixed) parameter inside the interval given by the statistic.

See also 'On the "Probable Error" of a Coefficient of Correlation Deduced from a Small Sample. Fisher 1921'On the "Probable Error" of a Coefficient of Correlation Deduced from a Small Sample.' Fisher 1921 in which Fisher demonstrated the difference of his method not being a Bayesian inverse probability.

Note again that likelihood is a density function forof a set of parameterparameters, but not a probability density function for aof that set of parameters.

Probability is used for something you can observe. E.g the probability that a dice rolls six. Likelihood is used for something that you can not observe, e.g. the hypothesis that a dice rolls six 1/6 of the time.

If you can understand and appreciate the comments from Fisher on the difference between inverse probability and the principle of likelihood you may wish to read further on differences betweenwithin frequentist methods.

Which is a work of 50 pages and difficult to summarize. But it deals with your questions on unbiasedness, explains the method of least squares (and difference with method of maximum likelihood), and specifically provides a treatment of confidence intervals (whichfrequentist interval are already not at all always that much the samesimilar, unique, let alone that the are the same as Bayesian intervals for flat priors).

Regarding the F-testRegarding the F-test it is not clear, what in the name of Laplace you think is wrong. If you like an early use you can look in 'Studies in crop variation. II. The manurial response of different potato varieties' 1923 Fisher and Mackenzie

This paper has the expression of anova in a recognizable linear model subdividing the sums of squares into between and within groups.

(in the test of the 1923 article the test consists of a comparison of differences between the logs of sample standard deviations with a calculated standard error for this difference that is determined by a sum of degrees of freedom $\frac{1}{2d_1} + \frac{1}{2d_2}$. Later works make this more sophisticated expressions leading to the F-distribution, such that it may diffuse the ideas that one may have about it. But in essence, without the technical juggling due to more exact distributions for small numbers, it's origin is much like a z-test).

You make a notion of the Likelihood as being a Bayesian expression with a flat prior, however while the mathematics coincide (when wrongly interpreted, since you may get P(x|a) = P(a|x), but they are not the same terms) the construction and meaning is different.

Likelihood is not meant to be a 'Bayesian probability based on flat, or uniformed, priors'. Likelihood is not even a probability and does not follow the rules of a probability (for instance you can not add up likelihood for different events, and the sum is not equal to one), it is only when you multiply it with a flat prior, that it becomes a probability, but then the meaning has changed as well.

Bayesian and frequentists methods are different tools:

By constructing a table of corresponding values, we may know as soon as T is calculated what is the fiducial 5 per cent, value of $\theta$, and that the true value of $theta$ will be less than this value in just 5 per cent, of trials. This then is a definite probability statement about the unknown parameter $\theta$, which is true irrespective of any assumption as to its a priori distribution.

a frequentist method makes a statement about the probability that an experiment will have the true value of a parameter (which may differ from experiment to experiment) inside the interval given by a statistic. This is not the be confused with the probability that a specific experiment will have the true value of the parameter inside the interval given by the statistic.

See also 'On the "Probable Error" of a Coefficient of Correlation Deduced from a Small Sample. Fisher 1921 in which Fisher demonstrated the difference of his method not being a Bayesian inverse probability.

Note again that likelihood is a density function for a set of parameter, but not a probability density function for a set of parameters.

Probability is used for something you can observe. E.g the probability that a dice rolls six. Likelihood is used for something that you can not observe, e.g. the hypothesis that a rolls six 1/6 of the time.

If you can understand and appreciate the comments from Fisher on the difference between inverse probability and the principle of likelihood you may wish to read further on differences between frequentist methods.

Which is a work of 50 pages and difficult to summarize. But it deals with your questions on unbiasedness, explains the method of least squares (and difference with method of maximum likelihood), and specifically provides a treatment of confidence intervals (which are not at all always that much the same, let alone the same as Bayesian intervals).

Regarding the F-test it is not clear, what in the name of Laplace you think is wrong. If you like an early use you can look in 'Studies in crop variation. II. The manurial response of different potato varieties' 1923 Fisher and Mackenzie

This paper has the expression of anova in a recognizable linear model.

You make a notion of the likelihood as being a Bayesian expression with a flat prior

However,

  1. while the mathematics coincide (when wrongly interpreted, since you may get P(x|a) = P(a|x), up to a constant, but they are not the same terms) the construction and meaning is different.

  2. Likelihood is not meant to be a 'Bayesian probability based on flat, or uniformed, priors'. Likelihood is not even a probability and does not follow the rules of probability distributions (for instance you can not add up likelihood for different events, and the integral is not equal to one), it is only when you multiply it with a flat prior, that it becomes a probability, but then the meaning has changed as well.

Bayesian and frequentist methods are different tools:

By constructing a table of corresponding values, we may know as soon as T is calculated what is the fiducial 5 per cent, value of $\theta$, and that the true value of $\theta$ will be less than this value in just 5 per cent, of trials. This then is a definite probability statement about the unknown parameter $\theta$, which is true irrespective of any assumption as to its a priori distribution.

  • a frequentist method makes a statement about the probability that an experiment (with random interval) will have the true value of a (possibly random) parameter inside the interval given by a statistic.
  • This is not the be confused with the probability that a specific experiment (with fixed interval) will have the true value of the (fixed) parameter inside the interval given by the statistic.

See also 'On the "Probable Error" of a Coefficient of Correlation Deduced from a Small Sample.' Fisher 1921 in which Fisher demonstrated the difference of his method not being a Bayesian inverse probability.

Note again that likelihood is a function of a set of parameters, but not a probability density function of that set of parameters.

Probability is used for something you can observe. E.g the probability that a dice rolls six. Likelihood is used for something that you can not observe, e.g. the hypothesis that a dice rolls six 1/6 of the time.

If you can understand and appreciate the comments from Fisher on the difference between inverse probability and the principle of likelihood you may wish to read further on differences within frequentist methods.

Which is a work of 50 pages and difficult to summarize. But it deals with your questions on unbiasedness, explains the method of least squares (and difference with method of maximum likelihood), and specifically provides a treatment of confidence intervals (frequentist interval are already not similar, unique, let alone that the are the same as Bayesian intervals for flat priors).

Regarding the F-test it is not clear, what in the name of Laplace you think is wrong. If you like an early use you can look in 'Studies in crop variation. II. The manurial response of different potato varieties' 1923 Fisher and Mackenzie

This paper has the expression of anova in a recognizable linear model subdividing the sums of squares into between and within groups.

(in the test of the 1923 article the test consists of a comparison of differences between the logs of sample standard deviations with a calculated standard error for this difference that is determined by a sum of degrees of freedom $\frac{1}{2d_1} + \frac{1}{2d_2}$. Later works make this more sophisticated expressions leading to the F-distribution, such that it may diffuse the ideas that one may have about it. But in essence, without the technical juggling due to more exact distributions for small numbers, it's origin is much like a z-test).

1
source | link

It appears to me as if you are considering a world of frequentists and Bayesians. That is not much nuanced. Like if you have to be the one or the other, or as if the methods applied are determined by some personal believes (rather than convenience and the specific problem and information at hand). I believe that this is a misconception based on current trends in calling oneself a frequentist or Bayesian, and also lots of statistical language may be confusing. Just try to have a group of statisticians explain p-value or confidence interval.

Some classical works may help you to understand frequentist inference. The classical works contain fundamental principles, are close to the heat of the discussion between proponents, and provide a background of the (practical) motivation and relevance at that time.

also, these classical works on frequentist methods, were written in a time when people mostly worked with Bayesian principles and mathematical calculation of probability (note that statistics is not always as if you are working on a typical mathematics problem with probabilities, the probabilities may be very ill-defined).

Frequentist probability is not inverse probability

'Inverse probability' Fisher 1930

You make a notion of the Likelihood as being a Bayesian expression with a flat prior, however while the mathematics coincide (when wrongly interpreted, since you may get P(x|a) = P(a|x), but they are not the same terms) the construction and meaning is different.

Likelihood is not meant to be a 'Bayesian probability based on flat, or uniformed, priors'. Likelihood is not even a probability and does not follow the rules of a probability (for instance you can not add up likelihood for different events, and the sum is not equal to one), it is only when you multiply it with a flat prior, that it becomes a probability, but then the meaning has changed as well.

Some interesting quotes from 'inverse probability' 1930 Fisher.

Bayesian and frequentists methods are different tools:

...there are two different measures of rational belief appropriate to different cases. Knowing the population we can express our incomplete knowledge of, or expectation of, the sample in terms of probability; knowing the sample we can express our incomplete knowledge of the population in terms of likelihood. We can state the relative likelihood that an unknown correlation is + 0.6, but not the probability that it lies in the range .595-.605.

Note that there is a certain probability statement, which a frequentist method provides.

By constructing a table of corresponding values, we may know as soon as T is calculated what is the fiducial 5 per cent, value of $\theta$, and that the true value of $theta$ will be less than this value in just 5 per cent, of trials. This then is a definite probability statement about the unknown parameter $\theta$, which is true irrespective of any assumption as to its a priori distribution.

a frequentist method makes a statement about the probability that an experiment will have the true value of a parameter (which may differ from experiment to experiment) inside the interval given by a statistic. This is not the be confused with the probability that a specific experiment will have the true value of the parameter inside the interval given by the statistic.

See also 'On the "Probable Error" of a Coefficient of Correlation Deduced from a Small Sample. Fisher 1921 in which Fisher demonstrated the difference of his method not being a Bayesian inverse probability.

In the former paper it was found, by applying a method previously developed, that the << most likely >> value of the correlation of the population was, numerically, slightly smaller than that of the sample. This conclusion was adversely criticized in Biometrica, apparently on the incorrect assumption that I had deduced it from Bayes theorem. It will be shown in this paper that when the sampling curves are rendered approximately normal, the correction I had proposed is equal to the distance between the population value and the mid-point of the sampling curve and is accordingly no more than the correction of a constant bias introduced by the method of calculation. No assumption as to a priori probability is involved.

and

...two radically distinct concepts have been confused under the name of << probability >> ...

that is probability and likelihood. See also the note on the end of Fishers article from 1921 in which he speaks more on the confusion.

Note again that likelihood is a density function for a set of parameter, but not a probability density function for a set of parameters.

Probability is used for something you can observe. E.g the probability that a dice rolls six. Likelihood is used for something that you can not observe, e.g. the hypothesis that a rolls six 1/6 of the time.

also, you might like Fisher's work in which he is much lighter in his opinion on Bayes theorem (still describing the differences). 'On the mathematical foundations of theoretical statistics' Fisher 1922 (especially section 6 'formal solution of problem of estimation')

More

If you can understand and appreciate the comments from Fisher on the difference between inverse probability and the principle of likelihood you may wish to read further on differences between frequentist methods.

'Outline of a Theory of Statistical Estimation Based on the Classical Theory of Probability' Neyman 1937

Which is a work of 50 pages and difficult to summarize. But it deals with your questions on unbiasedness, explains the method of least squares (and difference with method of maximum likelihood), and specifically provides a treatment of confidence intervals (which are not at all always that much the same, let alone the same as Bayesian intervals).

Regarding the F-test it is not clear, what in the name of Laplace you think is wrong. If you like an early use you can look in 'Studies in crop variation. II. The manurial response of different potato varieties' 1923 Fisher and Mackenzie

This paper has the expression of anova in a recognizable linear model.