Suppose we have continuous stream of data which length we cannot predict and discretize. Is there a type of neural network that can hold this stream and makes output based on the information stored in this stream?
Any neural network trained using some variant of online learning (e.g. stochastic gradient descent) will be able to do this. If the stream contains independent samples, then a feedforward network would work. If the stream contains sequence data with time dependencies that you want to model, then a recurrent network would be the tool of choice (trained using a method like backpropagation through time).