I have been using the probability of improvement acquisition function in my Bayesian Optimization program, but I've run into a problem because I am not optimizing the acquisition function that quickly. Currently, I'm using a PSO algorithm to find points from the acquisition functions, but as the matrices get larger it quickly becomes computationally burdensome.
The Probability of Improvement Function is:
PI(x) = P(f(x) ≥ f(x+)) = Φ (µ(x) - f(x+) / σ(x))
where f(x+) is the max value already found, µ(x) is the mean, σ(x) is the standard deviation, Φ () refers to the cumulative density function of a normal distribution.
More information about how the mean and standard deviation equations are derived can be found in this article: https://arxiv.org/pdf/1012.2599.pdf
My question really boils down to two parts. First, how can I analytically solve this equation with all the matrix math? I understand that the derivative of a cumulative density function is the probability density function, but as to finding the derivatives of the mean and standard deviation functions I don't understand how to attain them. Second, how does vector inputs change the answer to the first question? In my program I have multiple input parameters which I convert into vector inputs and then into scalars using a kernel so that the covariance matrix is a similarity measure between the parameter components of two given vectors. Therefore, if I were to solve the first question a get a given x that is the optimized point that x would have a to be a n-dimensional vector where n is the number of the parameters I have.