I don't think there can be one answer to all the deep learning models. WHich of the deep learning models are parametric and which are non-parametric and why?
Deep learning models are generally parametric - in fact they have a huge number of parameters, one for each weight that is tuned during training.
As the number of weights generally stays constant, they technically have fixed degrees of freedom. However, as there are generally so many parameters they may be seen to emulate non-parametric.
Gaussian processes (for example) use each observation as a new weight and as the number of points goes to infinity so too do the number of weights (not to be confused with hyper parameters).
I say generally because there are so many different flavours of each model. For example low rank GPs have a bounded number of parameters which are inferred by the data and I'm sure someone has been making some type of non-parametric dnn at some research group!
Deep learning models should not be considered parametric. Parametric models are defined as models based off an a priori assumption about the distributions that generate the data. Deep nets do not make assumptions about the data generating process, rather they use large amounts of data to learn a function that maps inputs to outputs. Deep learning is non-parametric by any reasonable definition.