I just started learning about neural networks and was wondering what a neural network with 2 hidden layers is able to express over a neural network with just 1 hidden layer. Specifically, I am trying to come up with an binary classification example with a decision boundary that can be expressed with 2 hidden layers but not one (assuming input is a point in the 2D space, ReLU activation for the hidden layers, and sigmoid activation for the output). The 1-layer network has 4 hidden neurons in its layer while the 2-layer network has 2 neurons each layer, so still 4 hidden neurons total. I've tried looking at concentric circles, enclosed shapes, and nonlinear boundaries but have no luck so far.

What is an example of a binary decision boundary that 2 hidden layers (2 neurons each layer) are able to capture but 1 hidden layer (4 neurons) isn't? Is this even possible?

Update: I edited my question to a limited number of neurons as that the universal approximation theorem does not apply.

I just started learning about neural networks and was wondering what a neural network with 2 hidden layers is able to express over a neural network with just 1 hidden layer (where number of neurons are limited). Specifically, I am trying to come up with an binary classification example with a decision boundary that can be expressed with 2 hidden layers but not one (assuming input is a point in the 2D space, ReLU activation for the hidden layers, and sigmoid activation for the output). The 1-layer network has 4 hidden neurons in its layer while the 2-layer network has 2 neurons each layer, so still 4 hidden neurons total. I've tried looking at concentric circles, enclosed shapes, and nonlinear boundaries but have no luck so far.

What is an example of a binary decision boundary that 2 hidden layers (2 neurons each layer) are able to capture but 1 hidden layer (4 neurons) isn't? Is this even possible?

Update: I edited my question to a limited number of neurons as that the universal approximation theorem does not apply.

I just started learning about neural networks and was wondering what a neural network with 2 hidden layers is able to express over a neural network with just 1 hidden layer. Specifically, I am trying to come up with an binary classification example with a decision boundary that can be expressed with 2 hidden layers but not one (assuming input is a point in the 2D space, ReLU activation for the hidden layers, and sigmoid activation for the output). I've tried looking at concentric circles, enclosed shapes, and nonlinear boundaries but have no luck so far.

What is an example of a binary decision boundary that 2 hidden layers are able to capture but 1 hidden layer isn't? Is this even possible?

I just started learning about neural networks and was wondering what a neural network with 2 hidden layers is able to express over a neural network with just 1 hidden layer. Specifically, I am trying to come up with an binary classification example with a decision boundary that can be expressed with 2 hidden layers but not one (assuming input is a point in the 2D space, ReLU activation for the hidden layers, and sigmoid activation for the output). The 1-layer network has 4 hidden neurons in its layer while the 2-layer network has 2 neurons each layer, so still 4 hidden neurons total. I've tried looking at concentric circles, enclosed shapes, and nonlinear boundaries but have no luck so far.

What is an example of a binary decision boundary that 2 hidden layers (2 neurons each layer) are able to capture but 1 hidden layer (4 neurons) isn't? Is this even possible?

Update: I edited my question to a limited number of neurons as that the universal approximation theorem does not apply.