Non-Bayesian predictive inference (apart from the SLR case) is a relatively recent field. Under the heading of "non-Bayesian" we can subdivide the approaches into those that are "classical" frequentist vs those that are "likelihood" based.
Classical Frequentist Prediction
As you know, the "gold standard" in frequentism is to achieve the nominal coverage under repeated sampling. For example, we want a 95% Confidence Region to contain the true parameter(s) in 95% of samples from the same underlying population. Or, we expect to commit Type I and II errors in a hypothesis test on average equal to $\alpha$ and $\beta$. Finally, and most germane to this question, we expect our 95% Prediction Interval to contain the next sample point 95% of the time.
Now, I've generally had issues with how classical PI's are presented and taught in most stats courses, because the overwhelming tendency is to interpret these as Bayesian posterior predictive intervals, which they are decidedly not. Most fundamentally, they are talking about different probabilities! Bayesian's make no claim on the repeated sampling performance of their quantities (otherwise, they'd be frequentists). Second, a Bayesian PI is actually accomplishing something more similar in spirit to a Classical Tolerance Interval than to a Classical Prediction Interval.
For reference: Tolerance Intervals need to be specified by two probabilities: The confidence and the coverage. The confidence tells us how often it is correct in repeated samples. The coverage tells us the minimum probability measure of the interval under the true distribution (as opposed to the PI, which gives the expected probability measure...again under repeated sampling). This is basically what the Bayesian PI is trying to do as well, but without any repeated-sampling claims.
So, the basic logic of the Stats 101 Simple Linear Regression is to derive the repeated sampling properties of the PI under the assumption of normality. Its the frequentist+Gaussian approach that is typically thought of as "classical" and taught in intro stats classes. This is based on the simplicity of the resulting calculations (see Wikipedia for a nice overview).
Non-gaussian probability distributions are generally problematic because they can lack pivotal quantities that can be neatly inverted to get an interval. Therefore, there is no "exact" method for these distributions, often because the interval's properties depend on the true underlying parameters.
Acknowledging this inability, another class of prediction arose (and of inference and estimation) with the likelihood approach.
Likelihood-based Inference
Likelihood-based approaches, like many modern statistical concepts, can be traced back to Ronald Fisher. The basic idea of this school is that, except for special cases, our statistical inferences are on logically weaker ground than when we are dealing with inferences from a normal distribution (whose parameter estimates are orthogonal), where we can make exact probability statements. In this view of inference, one should really avoid statements about probability except in the exact case, otherwise, one should make statements about the likelihood and acknowledge that one does not know the exact probability of error (in a frequentist sense).
Therefore, we can see likelihood as akin to Bayesian probability, but without the integrability requirements or the possible confusion with frequentist probability. Its interpretation is entirely subjective...although a likelihood ratio of 0.15 is often recommended for single parameter inference.
However, one does not often see papers that explicitly give "likelihood intervals". Why? It appears that this is largely a matter of sociology, as we have all grown accustomed to probability-based confidence statements. Instead, what you often see is an author referring to an "approximate" or "asymptotic" confidence interval of such and such. These intervals are largely derived from likelihood methods, where we are relying on the asymptotic Chi-squared distribution of the likelihood ratio in much the same way we rely on the asymptotic normality of the sample mean.
With this "fix" we can now construct "approximate" 95% Confidence Regions with almost as much logical consistency as the Bayesians.
From CI to PI in the Likelihood Framework
The success and ease of the above likelihood approach led to ideas about how to extend it to prediction. A very nice survey article on this is given here (I will not reproduce its excellent coverage). It can be traced back to David Hinkley in the late 1970's (see JSTOR), who coined the term. He applied it to the perennial "Pearson's Binomial Prediction Problem". I'll summarize the basic logic.
The fundamental insight is that if we include an unobserved data point, say $y$, in our sample, and then perform traditional likelihood inference on $y$ instead of a fixed parameter, then what we get is not just a likelihood function, but a distribution (unnormalized), since the "parameter" $y$ is actually random and therefore can be logically assigned a frequentist probability. The mechanics of this for this particular problem are reviewed in the links I provided.
The basic rules for getting rid of "nuisance" parameters to get a predictive likelihood are as follows:
- If a parameter is fixed (e.g., $\mu, \sigma$), then profile it out of the likelihood.
- If a parameter is random (e.g., other unobserved data or "random effects"), then you integrate them out (just like in Bayesian approach).
The distinction between a fixed and random parameter is unique to likelihood inference, but has connections to mixed effects models, where it seems that the Bayesian, frequentist, and likelihood frameworks collide.
Hopefully this answered your question about the broad area of "non-Bayesian" prediction (and inference for that matter). Since hyperlinks can change, I'll also make a plug for the book "In All Likelihood: Statistical Modeling and Inference using Likelihood" which discusses the modern likelihood framework at depth, including a fair amount of the epistemological issues of likelihood vs Bayesian vs frequentist inference and prediction.
References
- Prediction Intervals: Non-parametric methods. Wikipedia. Accessed 9/13/2015.
- Bjornstad, Jan F. Predictive Likelihood: A Review. Statist. Sci. 5
(1990), no. 2, 242--254. doi:10.1214/ss/1177012175.
http://projecteuclid.org/euclid.ss/1177012175.
- David Hinkley. Predictive Likelihood. The Annals of Statistics Vol. 7, No. 4 (Jul., 1979) , pp. 718-728 Published by: Institute of Mathematical Statistics
Stable URL: http://www.jstor.org/stable/2958920
- Yudi Pawitan. In All Likelihood: Statistical Modeling and Inference Using Likelihood. Oxford University Press; 1 edition (August 30, 2001). ISBN-10: 0198507658, ISBN-13: 978-0198507659. Especially Chapters 5.5-5.9, 10, and 16.