What can I do if I know that one feature in input vector is more important than others?

Question

I'm talking about neural network and reinforcement learning.

Let's say I have an agent and my input is the state of its environment and one feature of this input is not environment but internal state of the agent.

I mean like for example my agent is a hiking human with a backpack and he has his vision as input + weight of his backpack and if he sees something valuable in his vision, he can put it in his backpack.

He always walks on the same trope and sees the same things, his goal is to carry the most amount(and most valuable in total) of things to the end of this path, but he gets penalty or rewarded if his backpack is less than some weight or more than some weight.(I don't know, some_weight can change during his path, environment(vision, or some other feature in the vision) tells him what weight is good). And I have a single feature which tells him current weight of his backpack.

So my input is [env1, env2, env3, env4, backpack_weight].

How can I incorporate the knowledge that backpack_weight is a little bit different from env features and that it is very important?

It was just an approximation of my problem, my agent doesn't use convolutional layers, but it has an mlp layer for embedding I would say.

If I had a convolutional layer, I wouldn't feed backpack weight into it and just append it to the output vector of convolution, but I have a simple MLP layer instead(network is a deep network).

Should I feed this important feature after embedding layer(I just that it's embedding, it doesn't differ from others in any way. It's just the first layer in my network.)?

Or maybe it will be good to feed it to each layer in the network?

Or only to the last layer?

Edit: of course, he can throw things away from his backpack too. For completion of my example :)

Answer 1

Although I don't have the exact answer to your question, a general thought I had w.r.t. adding prior information about features to neural networks was:

Normally we use weight decay, which means unless the feature decreases error on the training set reduce the weight to zero.

What happens if we add a reverse of weight decay term to the weight of this feature, i.e. unless the training prediction error increases keep increasing the weight of this feature, is that kind of an prior knowledge added to the system?

Else does initializing the weight of this feature as a larger positive number than other features help?