How to make NN to output a higher value when one of the input variables is higher When a neural network has multiple variables as input, and I know from prior knowledge (e.g., laws of physics) that when the one important variable is high, the output value should also be high, I want to reflect this in the model.
However, when I train an NN, I leave it up to the model to decide whether the one important variable will be used and in which direction the output value will change.
What is the best way to add such prior knowledge when training the model?
 A: You could bypass the rest of the neural network and add this one feature back on at the end with a regression coefficient and return the regression coefficient, too (so that you can penalize it as desired). E.g. with PyTorch, you could do something like this in the forward step (I didn't try to train this, so some fiddling might be needed):
import torch
import torch.nn as nn

features = torch.randn(size=(100, 5))
outcome = torch.randn(size=(100, 1))

class ExampleNN(nn.Module):
    def __init__(self, insize, hidden, dropout):         
        super(ExampleNN, self).__init__()
        
        self.bn0 = nn.BatchNorm1d(num_features=insize-1, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)        
        self.lin1 = nn.Linear(in_features=insize-1, out_features=hidden)
        self.bn1 = nn.BatchNorm1d(num_features=hidden, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)        
        self.dropout1 = nn.Dropout(dropout)
        
        self.beta0 = nn.Parameter(torch.randn(size=(1,1)))        

    def forward(self, inputs):
        x = inputs[:,1:]
        
        # These are just some example transformations you might be doing
        x = self.bn0(x)
        x = self.lin1(x)
        x = self.bn1(x)
        x = self.dropout1(x)
         
        return inputs[:,0]*inputs[:,0] + x, self.beta0
    
model = ExampleNN(insize=5, hidden=100, dropout=0.2)

model(features)

Note that this specifies a linear relationship. You then apply your prior belief by penalizing the second returned value from the model (which is just the regression parameter) by whatever prior belief you have about that regression parameter.
Doing that of course prevents you from learning some more nuanced details on the relationship between the feature and outcome (and in fact, you need to specify the relationship). Another option would be to not do x = inputs[:,1:], but rather x = inputs and to additionally try to learn deviations from that specified relationship.
Update: Seeing the comments that it's a monotone effect of unknown functional shape, then some of the techniques to enforce monotonic effects used in regression modeling could be used e.g. see this paper.
