1
$\begingroup$

Some years ago I developed a nonlinear model and java fitting engine that performs well enough to be useful, but is definitely a hack. I would like to modernize and publish an open-source tool (offers to collaborate invited), but it’s terribly frustrating that I cannot find discussion of some basic ideas and techniques that I am sure must have been better developed by others. Maybe all I need is a few keywords or links to get me started. FWIW, I earn my living as a cow doctor.

I like M. Eppes explanation of Maximum Liklihood estimation of parameters for a nonlinear model (but I can’t follow all the notation and math). Next I want to incorporate prior knowledge of expected parameter values. To do that, I need a way to normalize the “power” of individual parameters. Then with expected values for mean and sd of each of the 4 parameters in my model, I’ll be able to fit even a data set with zero or one data point.

Cool, but not good enough. We know to expect significanrt error in the data points (in a biological sense - it’s not measurement error but biological variability outside the model). With one data point the best estimate is not a curve that goes through that point, but somewhere between the mean curve and the best-fit curve. How do we calculate “somewhere” to incorporate expected error in our priors?

I have a working kludge for this, and there is even a chance that it is mathematically correct, but I would like very much to understand it well enough to explain what I am doing.

$\endgroup$
2
  • 1
    $\begingroup$ The first thing I would do is to actually plot the data and try some obvious transformations like a log transform to see if a linear model might still be appropriate. Nowhere in your paper is the data plotted. $\endgroup$ – Frederik Ziebell Nov 1 '20 at 0:09
  • $\begingroup$ Frederik - Of course you are correct. I have done that - it is where I started. It's a good model, and because it does a good job of approximating nature it is fittable. I even made a slightly buggy online grapher. My question though is about understanding the Bayesian process of combining priors with sample data. The reason a linear model is inappropriate is that it is a mechanistic model - the parameter values hold meaning in the biological context, and estimating those biological parameters is important. $\endgroup$ – jehrlich Nov 1 '20 at 10:41
2
$\begingroup$

Ok, so from what I understand you have some prior information on some biological phenomenon and want to incorporate that prior information into a statistical model. That definitely sounds Bayesian.

I wouldn't recommend rolling your own estimator for this. Libraries like PyMC3 in python and tools like Stan (accessible through a variety of languages) have done a lot of this for you and are tested well. You can learn more about how these tools work by visiting their websites and reading their docs.

What you should probably focus on is how to do bayesian statistics, which really can't be taught through a stackexhange post. If you have difficulty following along with the math in your linked post, you are going to have a tough time with Bayes.

$\endgroup$
4
  • $\begingroup$ In "modernizing" I do envision using a library, possibly Jenetics. I need to define a cost function. To do that I would like to be confident that I am seeking an optimal solution in the MLE sense. I think this will depend on the design of the cost function. Certainly understanding Bayesian methods takes effort, but the problem can be stated in very intuitive terms, as I tried to do in my question. My frustration is in finding study materials that address this very practical problem without what seems to me very excessive layering of math. $\endgroup$ – jehrlich Nov 1 '20 at 10:50
  • 1
    $\begingroup$ @jehrlich I'm not sure what you're expecting to find, but Bayes and modelling is inherently mathematical. If you're looking for sources which abstract away the math and focus on implementation, Machine Learning and sources like Medium may be up your alley but you'd need more data than you've told us you have in order to take that approach. $\endgroup$ – Demetri Pananos Nov 1 '20 at 15:58
  • $\begingroup$ Is there some standard approach to this? If so what keywords or links will get me started. Given a nonlinear model with a single outcome variable and priors for mean and sd of parameters as well as mean and sd of outside-the-model effects, how do I formulate a cost function such that minimizing cost gives MLE parameter values. Or is it impossible? $\endgroup$ – jehrlich Nov 1 '20 at 23:12
  • $\begingroup$ @jehrlich Look up "Maximum a posteriori". Its a form of penalized maximum likelihood. That's probably a good place to start. $\endgroup$ – Demetri Pananos Nov 2 '20 at 4:35

Not the answer you're looking for? Browse other questions tagged or ask your own question.