2
$\begingroup$

After having inferred a statistical model for my physical system $Y=f(X_1, \dots X_n)$, I want to estimate the local sensitivity of each variable $X_i$ (one by one) around the point of maximum response.

All the methods I know are based on the analysis of the first derivatives along each variables. Being located at a maximum, all of these derivatives are flattened, these approaches are irrelevant.

When I plot $X_i \to f(X_1, \dots, X_n)$, with all $X_{j \neq i}$ fixed to $\arg \max f$, I can visualize the shape of the response, usually asymmetric bell curves, but I don't know how to quantify the sensitivity.

Bonus points for an approach including sensitivity of joint variables :)

$\endgroup$

1 Answer 1

1
$\begingroup$

If $f$ is 'smooth'*, then at a local maximum, all the directional derivatives will be zero.

* without getting distracted by technicalities like the precise condition because I think you understand this part perfectly well already

The usual definition of something like 'local sensitivity' is based on a first partial derivative with respect to the variable we're looking at.

If your definition of 'local sensitivity' is the usual one, then clearly they're all zero; indeed, that's entirely the point - in the very small, the function is insensitive to small changes in the variables.

It's telling you what you're asking it.

But they do change a tiny bit. Think about a quadratic in a single variable, one with a maximum point. What governs the rate at which that quadratic decreases from that maximum as $x$ moves away from its value there? If $f = ax^2 + bx+c$ (noting that $a$ must be negative), then $x^* = -\frac{b}{2a}$ is the argmax of $f$. Let $f^* = f(x^*)$. What's the value of $f$ at $x^*+dx$?

$f(x^*+dx) = f^* + \frac{df}{dx} + \frac{1}{2} \frac{d^2f}{dx^2}$ (in this particular case, exactly)

but at the maximum, $\frac{df}{dx}=0$, so

$f(x^*+dx) - f^* \approx \frac{1}{2} \frac{d^2f}{dx^2}|_{x^*} = a$

($a$ is negative, so this is showing a drop)

So it is with this problem. If you want to be able to compare the relative sizes of the changes, consider a Taylor expansion around the maximum, that is, something that describes $f$'s deviations from $f^*$, its value at the maximum, in terms of the values of the arguments that produce that maximum. Take as the argmax of $f$, $X^* = (X_1^*,...,X_n^*)$.

If the first order terms that govern the Taylor series are all zero, then the second derivative tells you about the behavior as you deviate from that maximum - how quickly $f$ decreases as you change each $X$ away from its value at the maximum.

So for example, if you're interested in how it changes as you change $X_1$, then $\frac{1}{2} \frac{\partial^2 f}{\partial X_1^2}|_{\mathbf{X}^*}$ tells you approximately the rate at which $f$ moves from $f^*$ as $X_1$ changes by a tiny amount from $X_1^*$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.