# How to input a continuous distribution to a neural network

I have simulated the relative frequency of a stochastic process by creating a very small grid say $1000$ by $1000$. The graph looks like this

Now I am trying to setup a regression model by matching each column/distribution - to a point of the desired output function. So there are $1000$-input nodes and $1$-output node. Here is a quick look

The input layer is very large and the network does not train well at all. I am not even sure if this is a learnable problem - although I have had a bit of success.

Is there a better way to input a continuous distribution to a neural network?

The data distribution seems like a natural normalizer and the most common way to describe data. I have been looking at VAEs (the reparameterization trick) but the distribution of the data is not known.

Any thoughts? Thank you

Edit

Per request, the expected value of a function $f(x)$ of a random variable $X_t$ with pdf $p(x)$ is the following

$$\mathbb{E}_x[f(X_t)] = \int f(x)p(x_t)dx$$

The random variable $X$ is dependent on time; hence, its probability density function changes over an observe period of time.

Suppose we can measure the probability density $p(x)$ and observe the quantity $\mathbb{E}_x[f(X_t)]$ over the same period. We want to set up a neural network to learn the function $f(x)$ which is a deterministic function i.e.: $\lambda x^2$. Note that the function is not dependent on time; hence, each input training set $(t,p(x)|t=s)$ and output $\mathbb{E}_x[f(X_t)|t=s]$ is an example of this unknown function. The goal is not to predict the next point in the time series.

Motivation

The motivation is similar to the VAE goal

In my example $f(x)=g(z)$ but in a supervised setting.

• What are you trying to achieve? Learn a network to model this function you just plotted? What exactly is the input and the output? – Jan Kukacka Aug 17 '18 at 9:04
• @JanKukacka Yes exactly - I want the network to learn the function on the right. The input is the evolving probability distribution of a random variable $X$ and the output is the desired expected value $$\mathbb{E}_x[f(X)] = \int f(x)p(x)dx$$ – Edv Beq Aug 17 '18 at 10:23
• @JanKukacka Actually, I want the network to learn the deterministic function $f(x)$ - this is more correct. – Edv Beq Aug 17 '18 at 10:55
• The function you plotted in the upper figure has two variables. Which of them is $x$ and what does the other axis represent? – Jan Kukacka Aug 17 '18 at 11:34
• @JanKukacka The two variables are $(t, p(x_t))$. So, the distribution evolves forward in time but it does not really matter. I do not want to model this as RNN - I want it as a regression model because I want the network to learn the deterministic function $f(x)$ not predict the time series. So the columns are time indexes which get shuffled anyways when training. – Edv Beq Aug 17 '18 at 11:39