It seems like every resource that discusses exponential families simply defines the family of distributions, explains why it's useful and then derives some of its properties.
I have only seen one resource which seems to derive the exponential family through geometry. Specifically, "Differential Geometry and Statistics" by Murray and Rice. I don't have the background to understand this exposition, but I still want to understand where the exponentially family comes from.
How can one arrive at the form exponential family of distributions from first principles without prior knowledge of what it looks like?
I'd like to know what question does the form of the exponential family answer?
The answer I'm looking for would follow along these lines:
- What distributions have X,Y,Z properties?
- Writing down assumptions based on this question, we'd "solve" for the set of distributions satisfying these properties.
- Arrive at an equation describing the set defined in step one which happens to be algebraically equivalent to the exponential family of distributions.