I have a question regarding process control with the use of multivariate regression.

The setup is as follows: say we have some data, representing the results of a plant process. Specifically, several continuous variables $\{x_1,\ldots,x_n\}$ represent inputs of the process (and we can control directly only $x_1, x_2, x_3$) and there is a continuous dependent variable $y$ being the result of this process. Our goal is to model the dependence and to control $y$ with the help of $x_1, x_2, x_3$. So, given real data, to model this dependence we can build the multiple regression with $y=\sum \alpha_i x_i$.

However, to control $y$ with $x_1, x_2, x_3$, it seems better to "reverse" the problem, i.e. to build multivariate regression: the independent variables (inputs) are now $\{x_4,\ldots,x_n,y\}$ and we predict the values of $x_1, x_2, x_3$ to be applied in order to obtain $y$.

And now for the main part: suppose there are some constraints on the inputs $x_1, x_2, x_3$, e.g. we may be forced to put $x_3$ or $\frac{x_1}{x_2}$ equal to some values. In other words, sometimes we might not be able to freely adjust the inputs in order to obtain given value of $y$, but still we want to be as close to $y$ as possible, so it seems like an optimization problem. The question: how such constraints could be introduced in the model?


Let me rephrase the problem by means of a particular constraint $x_1/x_2=u$. Given the model $(x_1, x_2, x_3) = f(x_4,\ldots,x_n, y)$, it has been obviously trained on samples with various values of $x_1/x_2$; for simplicity, suppose that a half of samples has $x_1/x_2\approx 1$ and the other half $x_1/x_2\approx 2$, and these ratios were set during production due to some process-specific constraints, such as availability of quantities $x_1, x_2$ through time.

Now, let's assume the tuple $(x_4,\ldots,x_n, y)$ is such that the model gives $f(x_4,\ldots,x_n, y)=(x_1, x_2, x_3)$ with $x_1/x_2\approx 1$, but due to practical reasons we insist on input $(x'_1, x'_2, x'_3)$ with $x'_1/x'_2=2$.

In that case, can we find $(x'_1, x'_2, x'_3)$ that is optimal, i.e. such that $(x'_1, x'_2, x'_3, x_4, \ldots, x_n)$ gives $y'$ that minimizes $(y-y')^2$? A wild guess would be to use a multiple regression $y=\sum \alpha_i x_i$ and then minimize an objective function $(y-y')^2$ w.r.t. $(x_1,x_2,x_3)$ with $x_1/x_2=2$, or perhaps there is another useful strategy?


1 Answer 1


Sounds like you might already have a model $(x_1,x_2,x_3)=f(x_4,\ldots,x_n,y;\hat\theta)$, so I'll just talk about the constraints.

If you have a constraint like $x_1/x_2 = u = \text{const}$, then you actually only need to model $(x_2,x_3) = f(\ldots)$, since you can recover $x_1$ with $x_1=ux_2$. Similarly if $x_3$ is fixed then this is even easier.

If you're constraints are more complicated than this (i.e. nasty and non-linear), you may be able to parameterize your problem, but failing that you could try nonlinear modelling with something like JuMP https://jump.dev/

  • $\begingroup$ If I got it right, you propose to train a new model each time we face a constraint. My idea is a little bit different: suppose the multivariate regression for $(x_1, x_2, x_3)$ has been trained on data with various ratios $x_1/x_2$. Then, based on that single model, I would like to find an optimal $(x_1, x_2, x_3)$ given some particular $x_1/x_2=u=\mathrm{const}$. Does this make sense? $\endgroup$
    – krzysiekb
    Aug 12, 2022 at 7:17
  • $\begingroup$ I was thinking you would bake all of your constraints into a single model, not train a new one with each constraint. If you are having to add new constraints frequently or constraints are changing frequently then I suppose that may not be practical. $\endgroup$
    – BenA
    Aug 12, 2022 at 7:42
  • $\begingroup$ I edited the question, hope it narrows down the problem. $\endgroup$
    – krzysiekb
    Aug 18, 2022 at 12:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.