Neural Nework and mathematical programming

Question

I am solving a scheduling problem where staff members cannot have overtime. For this problem, I have an integer programming model and solve it using the CPLEX solver. A simplified version of this model looks like the below:

$$\begin{align*} \min Z &= X C^\mathsf T\\ XB^\mathsf T &\geq b\\ X &\in Z_{\geq0}\end{align*} $$

Given the complexity of the problem, I would like to approximate the overtime component of the problem for each given solution $X$ (e.g., the number of scheduled patients). For this, I have built a simulation model, run it for several different potential values of $X$ (input values) and calculated the total overtime (output values). Now, I need a way to build a LINEAR approximation equation in $X$ and include it in the above optimization model. One way is to use regression analysis and fit an algebraic model to predict the overtime.

However, I would like to know whether I could use a neural network to derive a linear algebraic equation in $X$ and predict the overtime for different values of $X$? I have found this article, but it seems that the resulting equations are not linear due to the presence of activation functions. Note that I do not want to use the neural network to solve the problem. I just want to use the neural network to derive a linear algebraic equation in $X$ and include it as a constraint set in the above model.

Answer 1

You are right, the activation functions prevent the neural network (NN) from being a linear function, unless you only use linear activation functions, but then it would be again just a linear model. It's this nonlinearity in the activation functions which makes them perform so well in some situations.

If you want to obtain a linear model, you should fit a linear model. Even if you would find a way to use a NN and somehow apply some constraints that makes it learn a linear model, this wouldn't be better, and likely worse, than fitting a linear model.

Answer 2

We can use the ReLU activation function and derive a linear equation from a neural network. Further information is presented in this paper (doi: 10.1007/s10601-018-9285-6)