1
$\begingroup$

I have the following problem:

enter image description here

Here is my approach:
With the activation function: $F(x) = x^2 + 2x + 3$, we can calculate the activation of the two units of the second layer by: $a_1^2 = F(w_{13}\cdot x_1 + w_{23}\cdot x_2) = F(2\cdot 1 + (-3)\cdot (-1)) = F(5) = 38$
$a_2^2 = F(w_{14}\cdot x_1 + w_{24}\cdot x_2) = F(1\cdot 1 + 4\cdot (-1)) = F(-3) = 6$

With new inputs, we can now calculate the Output by:
$h(x) = F(w_{35}\cdot a_1^2 + w_{45}\cdot a_2^2)$ = $F(2\cdot 38 + (-1)\cdot 6) = F(70) = 5043$

I was wondering whether my approach to the problem was correct or not?

I would really appreciate any comments. Thank you all!

$\endgroup$
2
  • $\begingroup$ Please add the self-study tag & read its wiki. $\endgroup$
    – Sycorax
    Mar 13, 2023 at 13:07
  • $\begingroup$ yes what you wrote is correct $\endgroup$
    – Alberto
    Mar 13, 2023 at 23:47

1 Answer 1

0
$\begingroup$

Yes, what you wrote appears to be correct.

The final layer is just one neuron, and it has activation function $F$ applied to some values. Those values come from the values in the hidden neurons (call them $h_1$ and $h_2$) multiplied by their respective weights.

So far, this gives $F(w_{35}h_1 + w_{45}h_2)$.

You get $h_1$ from the input feature values times their respective weights, and then you apply the activation function $F$. Ditto for $h_2$.

$$ h_1 = F(w_{13}x_1 + w_{23}x_2)\\ h_2 = F(w_{14}x_1 + w_{24}x_2) $$

Finally, combine it all.

$$ F(w_{35}F(w_{13}x_1 + w_{23}x_2 + w_{45}F(w_{14}x_1 + w_{24}x_2)) $$

(Unrealted to the question, seeing it written out with this composition of functions, is it clear why there are a bunch of chain rule derivatives when you do the optimization calculus?)

When I run this in R software, I get the same $5043$ you got.

# Define the activation function
#
f <- function(x){
  return(x^2 + 2*x + 3)
}

# Define the weights
#
w13 <-  2
w23 <- -3
w14 <-  1
w24 <-  4
w35 <-  2
w45 <- -1

# Define the input feature values
#
x1 <-  1
x2 <- -1

# Calculate the values of the hidden-layer neurons
#
h1 <- f(w13*x1 + w23*x2)
h2 <- f(w14*x1 + w24*x2)

# Use the hidden-layer neurons to calculate the final output
#
f(w35*h1 + w45*h2)

(The variable F is taken as meaning FALSE in my software package, so I went with the lowercase f.)

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.