I have simulated the relative frequency of a stochastic process by creating a very small grid say $1000$ by $1000$. The graph looks like this

enter image description here

Now I am trying to setup a regression model by matching each column/distribution - to a point of the desired output function. So there are $1000$-input nodes and $1$-output node. Here is a quick look

enter image description here

The input layer is very large and the network does not train well at all. I am not even sure if this is a learnable problem - although I have had a bit of success.

Is there a better way to input a continuous distribution to a neural network?

The data distribution seems like a natural normalizer and the most common way to describe data. I have been looking at VAEs (the reparameterization trick) but the distribution of the data is not known.

Any thoughts? Thank you


Per request, the expected value of a function $f(x)$ of a random variable $X_t$ with pdf $p(x)$ is the following

$$\mathbb{E}_x[f(X_t)] = \int f(x)p(x_t)dx$$

The random variable $X$ is dependent on time; hence, its probability density function changes over an observe period of time.

Suppose we can measure the probability density $p(x)$ and observe the quantity $\mathbb{E}_x[f(X_t)]$ over the same period. We want to set up a neural network to learn the function $f(x)$ which is a deterministic function i.e.: $\lambda x^2$. Note that the function is not dependent on time; hence, each input training set $(t,p(x)|t=s)$ and output $\mathbb{E}_x[f(X_t)|t=s]$ is an example of this unknown function. The goal is not to predict the next point in the time series.


The motivation is similar to the VAE goal

enter image description here

In my example $f(x)=g(z)$ but in a supervised setting.

  • $\begingroup$ What are you trying to achieve? Learn a network to model this function you just plotted? What exactly is the input and the output? $\endgroup$ – Jan Kukacka Aug 17 '18 at 9:04
  • $\begingroup$ @JanKukacka Yes exactly - I want the network to learn the function on the right. The input is the evolving probability distribution of a random variable $X$ and the output is the desired expected value $$\mathbb{E}_x[f(X)] = \int f(x)p(x)dx$$ $\endgroup$ – Edv Beq Aug 17 '18 at 10:23
  • $\begingroup$ @JanKukacka Actually, I want the network to learn the deterministic function $f(x)$ - this is more correct. $\endgroup$ – Edv Beq Aug 17 '18 at 10:55
  • $\begingroup$ The function you plotted in the upper figure has two variables. Which of them is $x$ and what does the other axis represent? $\endgroup$ – Jan Kukacka Aug 17 '18 at 11:34
  • $\begingroup$ @JanKukacka The two variables are $(t, p(x_t))$. So, the distribution evolves forward in time but it does not really matter. I do not want to model this as RNN - I want it as a regression model because I want the network to learn the deterministic function $f(x)$ not predict the time series. So the columns are time indexes which get shuffled anyways when training. $\endgroup$ – Edv Beq Aug 17 '18 at 11:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.