How is an RNN (or any neural network) a parametric model? I'm going through this paper A Multi-Horizon Quantile Recurrent Forecaster. The authors state that: 

3.1. Loss Function
In Quantile Regression, models are trained to minimize the total Quantile Loss (QL):
$L_q(y,\hat{y})=q(y−\hat{y})_+ +(1−q)(\hat{y}−y)_+$
where $(·)_+ = max(0, ·)$. 
When q = 0.5, the QL is simply the Mean Absolute Error, and its minimizer is the median of the predictive distribution. Let K be the number
  of horizons of forecast, Q be the number of quantiles of
  interest, then the $K × Q$ matrix $\hat{Y} = [\hat{y}_{t+k}^{(q)} ]_{k,q}$ is the output of a parametric model $\boldsymbol{g(y_{:t}, x, θ)}$, e.g. an RNN. (emphasis mine; $x$ in $g(y_{:t}, x, θ)$ are the external regressors) 
The model parameters are trained to minimize the total loss, $\sum_t\sum_q\sum_kL_q(y_{t+k},\hat{y}_{t+k}^{(q)})$, where t iterates through all forecast creation times (FCTs). Depending on the problem, components of the sum can be assigned different weights, to highlight or discount different quantiles and horizons.

Why is this model considered "parametric" ? I thought neural networks where by definition non-parametric? 
Is it the fact that it is outputting a set of quantiles and not just a point forecast that makes it parametric? But even then - they are using the above defined Pinball Loss to calculate the quantiles, not a predefined distribution - so it still seems non-parametric to me. 
How is this approach considered parametric? 
 A: A neural network is defined by the weights on its connections, which are its parameters. It doesn't matter what data the network was trained upon, once you have a set of weights, you can throw away your training dataset without repercussion. If you want to classify a new sample, you can do it with only the parameterized network. Even if your training dataset is very large, you can still describe the network with exactly the same number of parameters as a small training set.
A k-nearest neighbor classifier, as an example on the other hand, is nonparametric because it relies on the training data to make any predictions. You need every training point to describe your classifier, there's no way to abstract it into a parameterized model. In essence, the number of "parameters" in this model grows with the number of training points you have. If you want to classify a new sample, you need the training data itself, because it cannot be summarized by a smaller set of parameters. A kNN classifier training on a large dataset will have more effective parameters than a kNN classifier trained on a small dataset.
The neural network is parametric because it uses a fixed number of parameters to build a model, independent of the size of the training data.
