One-hot encoding as raw inputs for deep learning models can find its applications in many domains, such as bioinformatics, NLP, chemistry and so on. Suppose we trained a neural network $f(x)$ with $x$ one-hot encoded. Now I want to evaluate the importance of each character based on the gradient $\partial f(x)/\partial x$ (e.g. saliency, inputxgrad, integrated gradients ...). When training $f(x)$, the gradient of loss function is well-defined on the network weights. Then primary question here is if $f(x)$ is differentiable w.r.t. $x$?

Strictly speaking, $f(x)$ is defined on binary values. Then for instance, in the following figure, a small deviation in the position of "T" would make no sense. So $\partial f(x)/\partial x$ is not well-defined, is that correct?

In the case of NLP, one-hot encoded words are first represented by embedding vectors of continuous values, e.g. word2vec. Then for a trained language model, for evaluating word contribution, we don't need to trace back to one-hot encoding but only to embedding vectors.

I haven't found similar discussions after a quick search. Is this trivial? Thanks a lot for your inputs!