Return to Question

I am having trouble reconciling (or maybe even understanding properly) the following issues:

We have a dataset. We hypothesize a functional form for probability density. Then we estimate the parameters of our proposed model with some procedure, such as MLE, MAP, OLS, or other more esoteric ones.

At this point the estimation procedure is in the cross-hairs. What is the objective? To lower the KL divergence of the whole model to the true model (MLE), to lower the KL divergence of the whole model to the true model by taking into consideration some subjective view (MAP), or are we actually trying to estimate each parameter of the model in an unbiased fashion with the lowest possible variance or maybe even minimum mean square error on one of the parameters? How can one goes on to do inference when the objective of estimation doesn't coincide with the errors/losses one will have when applying this model?

Moreover it seems we ignore the model error completely in this picture. In machine learning literature, there are at least some derived bounds, but in statistics literature this issue seems to be ignored (or at least at my level of reading).

In addition to all above, the Information Criteria tools available to us, such as AIC, BIC, TIC, etc. are targeted towards KL divergence, rather than our own objective. This leaves us with cross-validation to estimate the expected loss.

Thanks in advance

I am having trouble reconciling (or maybe even understanding properly) the following issues:

We have a dataset. We hypothesize a functional form for probability density. Then we estimate the parameters of our proposed model with some procedure, such as MLE, MAP, OLS, or other more esoteric ones.

At this point the estimation procedure is in the cross-hairs. What is the objective? To lower the KL divergence of the whole model to the true model (MLE), to lower the KL divergence of the whole model to the true model by taking into consideration some subjective view (MAP), or are we actually trying to estimate each parameter of the model in an unbiased fashion with the lowest possible variance or maybe even minimum mean square error on one of the parameters? How can one goes on to do inference when the objective of estimation doesn't coincide with the errors/losses one will have when applying this model?

Moreover it seems we ignore the model error completely in this picture. In machine learning literature, there are at least some derived bounds, but in statistics literature this issue seems to be ignored (or at least at my level of reading).

In addition to all above, the Information Criteria tools available to us, such as AIC, BIC, TIC, MIC, etc. are targeted towards KL divergence, rather than our own objective. This leaves us with cross-validation to estimate the expected loss.

Thanks in advance.

I am having trouble reconciling (or maybe even understanding properly) the following issues:

We have a dataset. We hypothesize a functional form for probability density. Then we estimate the parameters of our proposed model with some procedure, such as MLE, MAP, OLS, or other more esoteric ones.

At this point the estimation procedure is in the cross-hairs. What is the objective? To lower the KL divergence of the whole model to the true model (MLE), to lower the KL divergence of the whole model to the true model by taking into consideration some subjective view (MAP), or are we actually trying to estimate each parameter of the model in an unbiased fashion with the lowest possible variance or maybe even minimum mean square error on one of the parameters? How can one goes on to do inference when the objective of estimation doesn't coincide with the errors/losses one will have when applying this model?

Moreover it seems we ignore the model error completely in this picture. In machine learning literature, there are at least some derived bounds, but in statistics literature this issue seems to be ignored (or at least at my level of reading).

In addition to all above, the Information Criteria tools available to us, such as AIC, BIC, TIC, etc. are targeted towards KL divergence, rather than our own objective. This leaves us with cross-validation to estimate the expected loss.

Thanks

I am having trouble reconciling (or maybe even understanding properly) the following issues:

We have a dataset. We hypothesize a functional form for probability density. Then we estimate the parameters of our proposed model with some procedure, such as MLE, MAP, OLS, or other more esoteric ones.

At this point the estimation procedure is in the cross-hairs. What is the objective? To lower the KL divergence of the whole model to the true model (MLE), to lower the KL divergence of the whole model to the true model by taking into consideration some subjective view (MAP), or are we actually trying to estimate each parameter of the model in an unbiased fashion with the lowest possible variance or maybe even minimum mean square error on one of the parameters? How can one goes on to do inference when the objective of estimation doesn't coincide with the errors/losses one will have when applying this model?

Moreover it seems we ignore the model error completely in this picture. In machine learning literature, there are at least some derived bounds, but in statistics literature this issue seems to be ignored (or at least at my level of reading).

In addition to all above, the Information Criteria tools available to us, such as AIC, BIC, TIC, etc. are targeted towards KL divergence, rather than our own objective. This leaves us with cross-validation to estimate the expected loss.

Thanks in advance

I am having trouble reconciling (or maybe even understanding properly) the following issues:

We have a dataset. We hypothesize a functional form for probability density. Then we estimate the parameters of our proposed model with some procedure, such as MLE, MAP, OLS, or other more esoteric ones.

At this point the estimation procedure is in the cross-hairs. What is the objective? To lower the KL divergence of the whole model to the true model (MLE), to lower the KL divergence of the whole model to the true model by taking into consideration some subjective view (MAP), or are we actually trying to estimate each parameter of the model in an unbiased fashion with the lowest possible variance or maybe even minimum mean square error on one of the parameters? How can one goes on to do inference when the objective of estimation doesn't coincide with the errors/losses one will have when applying this model?

Moreover it seems we ignore the model error completely in this picture. In machine learning literature, there are at least some derived bounds, but in statistics literature this issue seems to be ignored (or at least at my level of reading).

Thanks

I am having trouble reconciling (or maybe even understanding properly) the following issues:

We have a dataset. We hypothesize a functional form for probability density. Then we estimate the parameters of our proposed model with some procedure, such as MLE, MAP, OLS, or other more esoteric ones.

At this point the estimation procedure is in the cross-hairs. What is the objective? To lower the KL divergence of the whole model to the true model (MLE), to lower the KL divergence of the whole model to the true model by taking into consideration some subjective view (MAP), or are we actually trying to estimate each parameter of the model in an unbiased fashion with the lowest possible variance or maybe even minimum mean square error on one of the parameters? How can one goes on to do inference when the objective of estimation doesn't coincide with the errors/losses one will have when applying this model?

Moreover it seems we ignore the model error completely in this picture. In machine learning literature, there are at least some derived bounds, but in statistics literature this issue seems to be ignored (or at least at my level of reading).

In addition to all above, the Information Criteria tools available to us, such as AIC, BIC, TIC, etc. are targeted towards KL divergence, rather than our own objective. This leaves us with cross-validation to estimate the expected loss.

Thanks