Using output of predictive model for optimisation I am wondering how to use the output of a predictive model to optimise the value of some independent variable to maximise some target under some constraints.
For example, borrowing an example from ISL, let's say I want to allocate some budget across different advertising channels (TV, Newspaper, Radio) to maximise sales of a particular product. Assuming I have access to historical data I can build a predictive model (say a linear regression with polynomial and interaction features) and use the model to assess how important each advertising channel is and by how much sales might increase for each dollar I spend.
Let's now say that I have a limited budget and some constraints such as spending at least a minimum amount on each of the three channels. How can I find the optimal value for each of the independent variables that will maximise sales given the constraints? Is the predictive model even necessary at this point or do we need a different technique?
 A: Basically, your predictive model attempts to fit a nonlinear function (not necessarily quite true but close enough in this case) to the data that you see. Assuming you can do this successfully, then you have a function or set of functions.  Let a be your spending on channel A, let b be your spending on channel B, etc. Let p(a,b,...) be the profit model (nonlinear function approximator for relationship between spending and profit). Here I use a general form assuming that the profit associated with each spending may be associated. If instead you would expect the profit associated with each spending channel to be the same regardless of the spending on other channels, this works out to something like p(a,b,...) = p(a) + p(b) + ....
maximize p(a,b,c)
subject to:
    a + b + c  < spending_limit

If your model is a simple linear equation (linear regression), you can solve this with a host of linear or nonlinear optimization techniques. If your model is differentiable, you can solve it with a number of gradient-based methods such as gradient descent. If your model is pseudo-differentiable/smooth, you can solve it with pseudo-gradient methods like finite difference approximation for gradient descent. Depending on the nature of the model, you have a variety of different optimization techniques available.
