0
$\begingroup$

A lot of ML models, such as neural networks, are Universal Function Approximators. But when evaluating ML models, we use usually metrics, such as MSE or accuracy, to assess the performance of a ML model.

Is there a convenient way to "create" an approximation of the function after having trained a ML model, with for example SK-Learn? E. g. if the data generating function is 3x³+0.5y²+1z+0.2xyz or whatever, I wonder if I could rebuild the function more or less from a trained ML model. I know that this is feasible for very easy functions with e. g. simple neural networks or SVMs. I know that this procedure has some caveats in higher dimensions (function will probably never the correct function (especially if relevant variables are missing), it is difficult with high dimensional data and potentially difficult to interpret, ...).

Nevertheless, I see scenarios where being able to present an approximation of the functional mapping could be beneficial beside assessing predictive performance.

$\endgroup$

1 Answer 1

1
$\begingroup$

I am not sure I understand the question correctly. You want to approximate the function approximated by a ML model with another parametric function? In what scenario would that be applicable?

As a general rule, don't be afraid of not using AI. If there are other established methods of modeling, use them.

To stay with the neural network example: In principle, a neural network is nothing else than a weighted, summed interconnection of different transfer functions (tanh, ReLu, etc.). Of course, one can look at the weights and activation of individual neurons and try to draw any conclusions from that.

However, in the case when one has trained a good model (good in the sense of good generalization ability), one uses the entire "capacity" of the network. I.e. there are no unnecessary weights. Otherwise these could be eliminated by regularization / pruning. So in this case you already have a very good model. To simplify it further will be difficult. And of course the question then arises, why one would want to do that, if one already has a well working model.

In general, however, it is not possible to draw any conclusions about the physical origins of the input-output behavior from the trained weights of the model. At least not yet - there is a whole research direction dealing with this question: explainable AI.

If one already has a rough idea of the system interactions, one can use physics-inspired neural networks. These predetermine the structure of known relationships and only individual parameters or the behavior of individual subsystems are learned.

$\endgroup$
2
  • $\begingroup$ Thank you for your answer! Yeah, I think you understood the question right. If e. g. the data generating function is 3x³+0.5y²+1z+0.2xyz or whatever, I wonder if I could rebuild the function more or less from a trained ML model. Maybe from models, that are known to be less black-box than NNs, such as Decision Trees or SVMs. Maybe you are right and in these scenarios, one should not use ML at all. $\endgroup$ Jan 18 at 17:13
  • $\begingroup$ Without prior knowledge of its structure, even a "simple" function like 3x³+0.5y²+1z+0.2xyz will be difficult to derive from a ML model. At least, I am not aware of any method that would allow such a thing. Depending on the input range, more and more parameters are required for the model as the nonlinearity increases. In this respect, this "simple" function is not so trivial in the end. $\endgroup$
    – ylpjört
    Jan 19 at 7:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.