You're talking about the dimension of the loss function, and the answer is that the input dimension of the loss function is how many parameters and biases there are, and the output dimension is $1$.
Let's focus on that first one by looking at an example with linear regression.
$$y
=
\beta_0 + \beta_1x_1+\beta_2x_2+\beta_3x_3
$$
Square loss is thus
$$
L\big(\beta_0, \beta_1, \beta_2, \beta_3\big) = \sum_{i=1}^n \Bigg[\bigg(y_i - \big( \beta_0 + \beta_1x_{i1}+\beta_2x_{i2}+\beta_3x_{i3} \big)\bigg)^2 \Bigg]
$$
That is a function from $\mathbb{R}^4 \rightarrow \mathbb{R}$, agreed? I would call that five dimensions: four for the regression parameters and one for the output loss.
A neural network is surprisingly simular to this. The estimated value is some composition of functions, and then the loss depends on the values of the estimated parameters.
$$
y = b_2 + w_2\text{ReLU}\big(b_0 + w_0x\big) + w_3\text{ReLU}\big(b_1 + w_1x\big)
$$
(I'll post that network later, but I think it would be a useful exercise to draw out what's going on. People appear quite eager to think of neural networks as being webs of circles and lines, yet there is real math going on.)
Then the loss is a function of the weights $w$ and biases $b$. Let's do absolute loss here.
$$
L\big(
b_0, b_1, b_2, w_0, w_1, w_2, w_3
\big)
=$$
$$
\sum_{i=1}^n \Bigg\vert y_i - \bigg(b_2 + w_2\text{ReLU}\big(b_0 + w_0x_i\big) + w_3\text{ReLU}\big(b_1 + w_1x_i\big)\bigg)\Bigg\vert
$$
This is a function $\mathbb{R}^7 \rightarrow \mathbb{R}$. I'd call that eight dimensions.