# What do the "in/out features" of a neural network mean?

I am working on a Reinforcement Learning problem in StableBaselines3, but I don't think that really matters for this question. SB3 is based on PyTorch.

So, below is a screenshot of my model architecture (after I wrote in code that I want 2 layers, each with 64 units/nodes):

This is a PPO algorithm in Reinforcement Learning, which is why there are two (identical) networks. Again, I don't think that's really relevant to the question.

You can see that the first layer has an input of 101 features (because I have 101 variables), but an output of 64 features. I'm not sure I understand how this works.

What does it mean when it says # of "in/out features"? If these are fully-connected dense layers (which I think they are), there are way more than 101 or 64 connections, right? Since each node is connect to every single node in the next layer?

So then, what does the 101 and 64 refer to?

A linear layer has a weight matrix $$W$$ and a bias vector $$b$$. (If you do not estimate a bias vector, then $$b=0$$.) The weight matrix has shape $$n\times m$$, i.e. $$n$$ rows and $$m$$ columns. The bias vector has shape $$m \times 1$$. The total number of parameters is $$nm + m$$ (if you don't use a bias, then $$m$$ of these parameters are fixed at 0, i.e. they are not trained).
Given a vector input $$x$$ with shape $$m \times 1$$, the output of a linear layer is given by $$f(x) = Wx + b$$, which is a vector of shape $$n \times 1$$.
So we know that there are $$m$$ inputs ("in features") to the linear layer $$f$$ and $$n$$ outputs ("out features").
• We know the number of output features is 64 in the first 2 layers of your example because the printout says out_features=64. Yes, linear layers connect all inputs to all outputs; to verify, write out each element of the result as sums and products of the inputs. Layer 1 has 101 inputs and 64 outputs, so it has $101 \times 64 + 64$ parameters with biases. Layer 2 has $64^2 + 64$ parameters with biases.