How do I control which feature to learn?

Question

Let me explain this with a simple example: predicting housing price using features such as number of bedrooms, number of bathrooms, etc.

Now that the market is changing every day, there's an 'underlying rule' that the house price is closely related to a * number of bedrooms + b. I know this rule and I want to incorporate this to my model. The parameters a and b are changing every day, but not changing within the day. My plan is to add an additional input feature, which is a * number of bedrooms + b, and train a and b together with other features.

Assume we have sufficient data, I want to develop a model that

1. train on housing prices every morning, predict some prices in the afternoon.

2. learn the parameters a and b while training, use them in predicting, then re-train them the other day. i.e. reset a and b at the end of each day.

3. for other features, I want to preserve the weights and train them further every morning.

Can I do this without using a separate linear regressor?

Answer 1

(This answer assumes you want to use neural networks)

A single perceptron actually learns a linear weight of your input:

$output = w*input + b$

Keeping this as a separate neural network allows you to reset the weights training at any point of whichever network you want. You can also define different optimizers with different learning rates and so on.

To assimilate this with your other neural network outcomes simply integrate wherever you want, for example:

• feed the outcome of this neural network as one of the inputs of another neural network (for example by concatenating the result to any other inputs you want to use)
• incorporate it with the output of another neural network (by summing, multiplying, concatenating,...)

By backpropagating the loss, you should be able to improve different parts of your setup as desired.