Gradient of Log Normalizing Constant - Does it have a name and do we know of any properties? Suppose $p(x) = \frac{\tilde{p}(x)}{Z}$ is a density function. I was wondering if the gradient of the log of the normalizing constant has a name and if we know any properties of it (e.g. maybe some properties similar to the score function)
$$
\nabla_x \log Z = \nabla_x \log \int_{\mathcal{X}} \tilde{p}(x) dx 
$$
 A: This answer assumes that you are interested in computing $\nabla_{\theta} \log Z$ rather than $\nabla_{x} \log Z$.
Without foreclosing the possibility that there may exist other results relevant to your question, a key result that arises is when $p(x)$ is in the exponential family.
In this case, $\log Z$ is referred to as the log-partition function or cumulant function. And the main result is that its gradient $\nabla_{\theta} \log Z$ is the expectation of the sufficient statistics. Furthermore, higher-order moments of the sufficient statistics can be attained by taking higher order derivatives of the log-partition function with respect to the parameters $\theta$.
Addressing the comment (which has now been edited) that:

[The gradient of the log of the normalizing constant] doesn't have a special name but it's the expectation of the sufficient statistics.

To prevent the risk of confusion, the result is that $\nabla_{\theta} \log Z$ is the expectation of the sufficient statistics, and not $\nabla_{x} \log Z$, which has no meaning I am aware of.
Furthermore, the result concerning $\nabla_{\theta} \log Z$ holds only when $p(x)$ is in the exponential family, and is not in general appropriate to assume when $p(x)$ is some arbitrary density function not in the exponential family.
In response to:

Can we say that $\nabla_{\theta} \log Z = \frac{1}{Z}\nabla_{\theta} \int_{\mathcal{X}} \tilde{p}(x) dx = \frac{1}{Z}\int_{\mathcal{X}} \nabla_{\theta} \tilde{p}(x) dx$?

Again, if $p(x)$ is in the exponential family, then interchanging the order of differentiation and integration is appropriate i.e. the operators can commute and the equality holds without further thought.
However, when $p(x)$ is not in the exponential family, and is some arbitrary density, then whether the derivative and integration operators are commutative depends on the range of integration and hence on the support of $X$. In the case where the range of integration is finite, you can appeal to Leibniz integral rule under certain conditions to justify this interchange.
In the case where $p(x)$ is an arbitrary density and the range of integration is infinite, the interchange of differentiation and integration operators are governed by Lebesgue's dominated convergence theorem. This theorem requires the machinery of measure theory and real analysis to rigorously state and to invoke. There are simpler corollaries to assess this without recourse to this machinery, and these are stated in section 2.4 of Statistical Inference by Casella and Berger (2004).
