If I understand your question correctly, you want to "shut off" one of the input neurons of a ANN at training time and then turn it back on at inference time correct?

Well, this question can be answered by simply understanding how a ANN works. Let's just use a simple dense feed forward network as an example. This neural network will take each input feature vector multiply it with a set of weights, add some biases, and then compute the 2nd layer activations. It then takes those 2nd layer activations and does the same thing to get the 3rd layer activations and so on and so forth until the final output activations are calculated. All neurons in the 2nd layer are connected to every neuron in the first layer with a specific weight $w_{ij}$ where $i$ denotes the first layer's neuron and $j$ denotes the second layer's neuron (let's ignore biases for now). This weight matrix is generally initialized independently of the inputs.

So what happens if you set one of the features to 0? We'll you'll have a set of weights $w_{\hat{i}j}$ (where $\hat{i}$ denotes the specific feature you've "turned off") that will never be updated. Why will these weights never be updated? Well because the feature is always 0 so backprop will always find 0 gradient with respect to these weights. I will not show a mathematical proof here, but it's very simple to understand why this is the case. No matter what all the $w_{\hat{i}j}$ are the loss function will never be affected because they will always be multiplied by a 0 and as such there is never a gradient with respect to this set of weights.

So at inference time, what you have is a set of weights $w_{\hat{i}j}$ which are untrained. Now, if you initialized these weights to 0 when you began training, nothing would change. They would stay 0 throughout training, and it would be like if that feature never existed. Even if you put that feature back in during inference time, it will always be multiplied by a set of 0's and so it won't matter. However, if you randomly initialized $w_{\hat{i}j}$ as would be generally the case (you don't want to initialize all weight in a neural network to 0), what adding the feature back in at inference time will do is add a bit of random statistical noise to your neural network. The size of this effect depends on details of how the weights were initialized and how strongly your neural network's outputs depend on those weights. Since generally you initialize the weights randomly, there shouldn't be any bias to this statistical noise, it should just be like adding a little bit of white noise to your network. However, one could conceive of a situation where one maliciously initializes weights so that bias is introduced...

It's a bit hard to understand the reasoning behind why anyone would want to do this.

If I understand your question correctly, you want to "shut off" one of the input neurons of a ANN at training time and then turn it back on at inference time correct?

Well, this question can be answered by simply understanding how a ANN works. Let's just use a simple dense feed forward network as an example. This neural network will take each input feature vector multiply it with a set of weights, add some biases, and then compute the 2nd layer activations. It then takes those 2nd layer activations and does the same thing to get the 3rd layer activations and so on and so forth until the final output activations are calculated. All neurons in the 2nd layer are connected to every neuron in the first layer with a specific weight $w_{ij}$ where $i$ denotes the first layer's neuron and $j$ denotes the second layer's neuron (let's ignore biases for now). This weight matrix is generally initialized independently of the inputs.

So what happens if you set one of the features to 0? Well you'll have a set of weights $w_{\hat{i}j}$ (where $\hat{i}$ denotes the specific feature you've "turned off") that will never be updated. Why will these weights never be updated? Well because the feature is always 0 so backprop will always find 0 gradient with respect to these weights. I will not show a mathematical proof here, but it's very simple to understand why this is the case. No matter what all the $w_{\hat{i}j}$ are the loss function will never be affected because they will always be multiplied by a 0 and as such there is never a gradient with respect to this set of weights.

So at inference time, what you have is a set of weights $w_{\hat{i}j}$ which are untrained. Now, if you initialized these weights to 0 when you began training, nothing would change. They would stay 0 throughout training, and it would be like if that feature never existed. Even if you put that feature back in during inference time, it will always be multiplied by a set of 0's and so it won't matter. However, if you randomly initialized $w_{\hat{i}j}$ as would be generally the case (you don't want to initialize all weight in a neural network to 0), what adding the feature back in at inference time will do is add a bit of random statistical noise to your neural network. The size of this effect depends on details of how the weights were initialized and how strongly your neural network's outputs depend on those weights. Since generally you initialize the weights randomly, there shouldn't be any bias to this statistical noise, it should just be like adding a little bit of white noise to your network. However, one could conceive of a situation where one maliciously initializes weights so that bias is introduced...

It's a bit hard to understand the reasoning behind why anyone would want to do this.

If I understand your question correctly, you want to "shut off" one of the input neurons of a ANN at training time and then turn it back on at inference time correct?

Well, this question can be answered by simply understanding how a ANN works. Let's just use a simple dense feed forward network as an example. This neural network will take each input feature vector multiply it with a set of weights, add some biases, and then compute the 2nd layer activations. It then takes those 2nd layer activations and does the same thing to get the 3rd layer activations and so on and so forth until the final output activations are calculated. All neurons in the 2nd layer are connected to every neuron in the first layer with a specific weight $w_{ij}$ where $i$ denotes the first layer's neuron and $j$ denotes the second layer's neuron (let's ignore biases for now). This weight matrix is generally initialized independently of the inputs.

So what happens if you set one of the features to 0? We'll you'll have a set of weights $w_{\hat{i}j}$ (where $\hat{i}$ denotes the specific feature you've "turned off") that will never be updated. Why will these weights never be updated? Well because the feature is always 0 so backprop will always find 0 gradient with respect to these weights. I will not show a mathematical proof here, but it's very simple to understand why this is the case. No matter what all the $w_{\hat{i}j}$ are the loss function will never be affected because they will always be multiplied by a 0 and as such there is never a gradient with respect to this set of weights.

So at inference time, what you have is a set of weights $w_{\hat{i}j}$ which are untrained. Now, if you initialized these weights to 0 when you began training, nothing would change. They would stay 0 throughout training, and it would be like if that feature never existed. Even if you put that feature back in during inference time, it will always be multiplied by a set of 0's and so it won't matter. However, if you randomly initialized $w_{\hat{i}j}$ as would be generally the case (you don't want to initialize all weight in a neural network to 0), what adding the feature back in at inference time will do is add random statistical noise to your neural network. The size of this effect depends on details of how the weights were initialized and how strongly your neural network's outputs depend on those weights.

If I understand your question correctly, you want to "shut off" one of the input neurons of a ANN at training time and then turn it back on at inference time correct?

Well, this question can be answered by simply understanding how a ANN works. Let's just use a simple dense feed forward network as an example. This neural network will take each input feature vector multiply it with a set of weights, add some biases, and then compute the 2nd layer activations. It then takes those 2nd layer activations and does the same thing to get the 3rd layer activations and so on and so forth until the final output activations are calculated. All neurons in the 2nd layer are connected to every neuron in the first layer with a specific weight $w_{ij}$ where $i$ denotes the first layer's neuron and $j$ denotes the second layer's neuron (let's ignore biases for now). This weight matrix is generally initialized independently of the inputs.

So what happens if you set one of the features to 0? We'll you'll have a set of weights $w_{\hat{i}j}$ (where $\hat{i}$ denotes the specific feature you've "turned off") that will never be updated. Why will these weights never be updated? Well because the feature is always 0 so backprop will always find 0 gradient with respect to these weights. I will not show a mathematical proof here, but it's very simple to understand why this is the case. No matter what all the $w_{\hat{i}j}$ are the loss function will never be affected because they will always be multiplied by a 0 and as such there is never a gradient with respect to this set of weights.

So at inference time, what you have is a set of weights $w_{\hat{i}j}$ which are untrained. Now, if you initialized these weights to 0 when you began training, nothing would change. They would stay 0 throughout training, and it would be like if that feature never existed. Even if you put that feature back in during inference time, it will always be multiplied by a set of 0's and so it won't matter. However, if you randomly initialized $w_{\hat{i}j}$ as would be generally the case (you don't want to initialize all weight in a neural network to 0), what adding the feature back in at inference time will do is add a bit of random statistical noise to your neural network. The size of this effect depends on details of how the weights were initialized and how strongly your neural network's outputs depend on those weights. Since generally you initialize the weights randomly, there shouldn't be any bias to this statistical noise, it should just be like adding a little bit of white noise to your network. However, one could conceive of a situation where one maliciously initializes weights so that bias is introduced...

It's a bit hard to understand the reasoning behind why anyone would want to do this.