2
$\begingroup$

Commonly when modelling biological systems, some parameters may be from elsewhere or previous modelling fits, and are not being investigated in the current model.

These seem to be equivalent to the ML hyperparameters, in that they are stated prior to model training and are not the purpose of modelling the dataset.

I'm interested if this could be an appropriate use of terminology, or if this use of hyperparameter (i.e. not the Bayesian hyperparameter) is only applicable to ML.

edit:

An example of non-ML modelling - a set of ODEs/PDEs characterising dynamic systems of cellular interactions, where the form of the model is informed by chemical and biological understanding of the system.

A hyperparameter in this context could be one governing the growth of a tumour, taken from a previous experiment, when the current model is based upon experimental data that describe tissue oxygenation over time as the tumour grows.

$\endgroup$
2
  • $\begingroup$ Welcome to Cross Validated! Perhaps it'd help to edit your question to give a specific example of "non-ML modelling". At any rate I'd be more inclined to call something a hyperparameter if it's used in training the model but not in applying the model. For example any shrinkage hyperparameters used in training a regression model aren't the business of a biologist interpreting the final model or making predictions with it. $\endgroup$
    – Scortchi
    Commented Apr 10, 2022 at 7:56
  • $\begingroup$ Thank you for your comment, I've made the edit. I'm glad to have found Cross Validated, I think it's exactly StackExchange community I've been looking for! It sounds like the terminology may not be appropriate for this scenario - I'm new to this field and still getting my head around nuance, thank you for your help. $\endgroup$
    – Kell
    Commented Apr 10, 2022 at 11:01

1 Answer 1

5
$\begingroup$

@kjetil's put it well in our tag wiki:

"A parameter that is not strictly for the statistical model (or data generating process), but a parameter for the statistical method. It could be a parameter for: a family of prior distributions, smoothing, a penalty in regularization methods, or an optimization algorithm."

So the values of hyperparameters that led to a model are of no direct relevance when it comes to interpreting or predicting from the model; the values of parameters on the other hand are what the model is, the claims it makes about the phenomenon it's a model of.

In your example, & under this view, the tumour growth rate is then a parameter, albeit "fixed", "constant", or "known". It's a part of your model, not merely a part of how you got to your model.

I don't know where you draw the line between trad. stats & M.L. models, but hyperparameters can be certainly be employed in the training of regression models. With elastic nets & c. they specify the amount & kind of shrinkage; with stepwise regression they specify the probabilities for entry & exit of predictors.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.