# How is an RNN (or any neural network) a parametric model?

I'm going through this paper A Multi-Horizon Quantile Recurrent Forecaster. The authors state that:

3.1. Loss Function

In Quantile Regression, models are trained to minimize the total Quantile Loss (QL):

$$L_q(y,\hat{y})=q(y−\hat{y})_+ +(1−q)(\hat{y}−y)_+$$

where $$(·)_+ = max(0, ·)$$.

When q = 0.5, the QL is simply the Mean Absolute Error, and its minimizer is the median of the predictive distribution. Let K be the number of horizons of forecast, Q be the number of quantiles of interest, then the $$K × Q$$ matrix $$\hat{Y} = [\hat{y}_{t+k}^{(q)} ]_{k,q}$$ is the output of a parametric model $$\boldsymbol{g(y_{:t}, x, θ)}$$, e.g. an RNN. (emphasis mine; $$x$$ in $$g(y_{:t}, x, θ)$$ are the external regressors)

The model parameters are trained to minimize the total loss, $$\sum_t\sum_q\sum_kL_q(y_{t+k},\hat{y}_{t+k}^{(q)})$$, where t iterates through all forecast creation times (FCTs). Depending on the problem, components of the sum can be assigned different weights, to highlight or discount different quantiles and horizons.

Why is this model considered "parametric" ? I thought neural networks where by definition non-parametric?

Is it the fact that it is outputting a set of quantiles and not just a point forecast that makes it parametric? But even then - they are using the above defined Pinball Loss to calculate the quantiles, not a predefined distribution - so it still seems non-parametric to me.

How is this approach considered parametric?

• I wonder if "parametric" just means "having parameters which are learned by some training procedure" in this usage. – Sycorax says Reinstate Monica Nov 28 '18 at 18:47
• @Sycorax quite possible, but then what's with the $\theta$ in $g(y_{:t}, x, θ)$ ? – Skander H. Nov 28 '18 at 18:52
• I think $\theta$ are those learned parameters. – Sycorax says Reinstate Monica Nov 28 '18 at 18:53
• Can you provide a reference as to where its been referred to as being non-parametric? NNs typically have 1000s+ parameters (weights). I think of something like Gaussian Processes/ Bayesian models to be non-parametric. Regardless, personally, the argument of parametric vs non-parametric is fruitless. – sachinruk Aug 12 '19 at 0:40