Which properties yield the exponential family of distributions?

It seems like every resource that discusses exponential families simply defines the family of distributions, explains why it's useful and then derives some of its properties.

I have only seen one resource which seems to derive the exponential family through geometry. Specifically, "Differential Geometry and Statistics" by Murray and Rice. I don't have the background to understand this exposition, but I still want to understand where the exponentially family comes from.

How can one arrive at the form exponential family of distributions from first principles without prior knowledge of what it looks like?

I'd like to know what question does the form of the exponential family answer?

1. What distributions have X,Y,Z properties?
2. Writing down assumptions based on this question, we'd "solve" for the set of distributions satisfying these properties.
3. Arrive at an equation describing the set defined in step one which happens to be algebraically equivalent to the exponential family of distributions.
• From a practical point of view I would guess that people realized that presumably different distributions have astonishingly similar properties so that they searched for a 'common root cause' of these properties. Then they started to 'fuse' single pairs of these 'presumably different' distributions and figured out that the 'root cause' for the similarities is that they surprisingly arise from the same family just with different parameters. So they continued in that way and fused and fused until they arrived at the stage today: the exponential family. It is consistent in an important... Feb 11, 2019 at 21:54
• I don't understand the motivation to write this all as a comment Feb 11, 2019 at 22:15
• It outgrew as I was going, sorry ;-) I often start with a comment in order to get a better understanding of the question... Feb 11, 2019 at 23:04
• I gave one answer to a more specific version of this question at stats.stackexchange.com/questions/32103/…. It invokes the "Principle of Mathematical Laziness" in a maximum likelihood setting. It's not the only possible answer because your question is more general.
– whuber
Feb 11, 2019 at 23:47
• @whuber are you saying that a random variable is in the exponential family if and only it has a closed form solution of its MLE? Feb 12, 2019 at 4:10