Is this feature extraction? I have a data set consisting of a large number of univariate time series.  I have empirical reasons to expect that each time series can be well described by a function with a small number of parameters, but the parameters can vary across time series.
I have no theoretical reason to prefer any particular parameterization of the function.  I'm concerned that choosing a parameterization myself could result in bias in the subsequent analysis.  I'm wondering if it is possible to extract or visualize an appropriate "shape" or "type" of function directly from the data.  One of the features I expect to differ across time series is essentially a delay, so that the time series all have similar shapes but different temporal offsets and amplitudes.
This seems like a feature extraction problem in dimensionality reduction - am I on the right track?  I'm not sure how something like kernel PCA would deal with the delay aspect.  Can anyone recommend good references?
 A: Very interesting question!
Any answer would be guess unless you tell what kind of time-series do you have and show us a plot but anyways:


*

*Phase-Space Reconstruction is still far better than any other algorithm to deal with delays and recurrence within a time-series

*If your assumption about delays is correct, then Granger Causality gives you much information about the relation of those delays.

*Recurrent NNs and TDNNs are also for this problem but don't use them if you need the model. They are just computational black-boxes.

*Auto-correlation will also talk about "how time-series remember itself" but if you use phase-space embedding you will use them anyways.


Hope it helps :)
A: I think the use-case described: "empirical reasons to expect that each time series can be well described by a function with a small number of parameters, but the parameters can vary across time series" fits naturally within the context of functional principal component analysis (FPCA).
Through FPCA we are able to define principal modes of variations that are defined as functions over a continuum (eg. time) rather than a multi-dimensional discretised space (like it is typically done in standard PCA). Given that we will zero-centred our data by subtracting their mean, we can then estimate their covariance and through that get their principal modes of variations in the form of the eigenfunctions of that covariances.
To draw a direct analogy with the phraseology in the post: The FPC scores (ie. the projections of the original sample over the axis system defined by the eigenfunctions) can be use the "small number of parameters" to describe each functions and the eigenfunctions themselves are "extract(ed) and visualize an appropriate "shape" or "type" of function directly from the data". Notice that FPCA is a non-parametric technique so we do not need to worry about an exact parametric form for the functions of the sample or the derived "features"/eigenfunctions of the sample.
CRAN has a whole task view on the analysis of functional data.
