Calculate the output of a Neural Network

Question

I have the following problem:

Here is my approach:
With the activation function: $F(x) = x^2 + 2x + 3$, we can calculate the activation of the two units of the second layer by: $a_1^2 = F(w_{13}\cdot x_1 + w_{23}\cdot x_2) = F(2\cdot 1 + (-3)\cdot (-1)) = F(5) = 38$
$a_2^2 = F(w_{14}\cdot x_1 + w_{24}\cdot x_2) = F(1\cdot 1 + 4\cdot (-1)) = F(-3) = 6$

With new inputs, we can now calculate the Output by:
$h(x) = F(w_{35}\cdot a_1^2 + w_{45}\cdot a_2^2)$ = $F(2\cdot 38 + (-1)\cdot 6) = F(70) = 5043$

I was wondering whether my approach to the problem was correct or not?

I would really appreciate any comments. Thank you all!

Answer 1

Yes, what you wrote appears to be correct.

The final layer is just one neuron, and it has activation function $F$ applied to some values. Those values come from the values in the hidden neurons (call them $h_1$ and $h_2$) multiplied by their respective weights.

So far, this gives $F(w_{35}h_1 + w_{45}h_2)$.

You get $h_1$ from the input feature values times their respective weights, and then you apply the activation function $F$. Ditto for $h_2$.

$$ h_1 = F(w_{13}x_1 + w_{23}x_2)\\ h_2 = F(w_{14}x_1 + w_{24}x_2) $$

Finally, combine it all.

$$ F(w_{35}F(w_{13}x_1 + w_{23}x_2 + w_{45}F(w_{14}x_1 + w_{24}x_2)) $$

(Unrealted to the question, seeing it written out with this composition of functions, is it clear why there are a bunch of chain rule derivatives when you do the optimization calculus?)

When I run this in R software, I get the same $5043$ you got.

# Define the activation function
#
f <- function(x){
  return(x^2 + 2*x + 3)
}

# Define the weights
#
w13 <-  2
w23 <- -3
w14 <-  1
w24 <-  4
w35 <-  2
w45 <- -1

# Define the input feature values
#
x1 <-  1
x2 <- -1

# Calculate the values of the hidden-layer neurons
#
h1 <- f(w13*x1 + w23*x2)
h2 <- f(w14*x1 + w24*x2)

# Use the hidden-layer neurons to calculate the final output
#
f(w35*h1 + w45*h2)

(The variable F is taken as meaning FALSE in my software package, so I went with the lowercase f.)