I have the following question: Can Bayesian Models be used to Compensate for Small Datasets?
Suppose we have a linear regression problem (e.g. predict the age of giraffes based on their height and weight) where we were not able to collect sufficient data, but we have have access to historical information from similar problems that we believe might be useful for defining the priors within Bayes Law. I tried to manually write the estimation equations for a Bayesian Linear Regression problem:
In this type of problem, suppose we were only able to collect measurements on a very small number of giraffes - could the Bayesian Priors in theory (if they indeed happen to accurately reflect the real data) be used to compensate for the small datasets and serve to "push" the estimates of the linear regression model towards more realistic values?
In a Frequentist setting, we are limited by the data we collect: if our data has poor quality (e.g. missing data, inaccurate data, small data), the model is almost guaranteed to suffer. But as far as I understand, perhaps Bayesian Models might be able to overcome this problem by strategically exploiting historical information from previous studies (e.g. meta-analysis) and turning them into priors?