I was looking at Open Ai's actor-critic code and noticed that some of the neural network's weights are shared
class Policy(nn.Module): def __init__(self): super(Policy, self).__init__() self.affine1 = nn.Linear(4, 128) self.action_head = nn.Linear(128, 2) self.value_head = nn.Linear(128, 1) self.saved_actions =  self.rewards =  def forward(self, x): x = F.relu(self.affine1(x)) action_scores = self.action_head(x) state_values = self.value_head(x) return F.softmax(action_scores, dim=-1), state_values
I am not sure why this is possible, because, theoretically,
The actor should calculate a function
from states to actions
The critic should calculate a function
from states to reward
These are two different functions, thus I would expect separate weights.
The only reason I came up with was a projection of exterior information on the problem, which is "Embedding the state".
If that is the case, then who is to say it makes sense to use he same state embedding for both networks?
Why does weight sharing in actor critic make sense?