6
$\begingroup$

I was asked this question during a test and couldn't figure out the answer:

  • You have a set of curves against time $X_i(t)$ that you want to use as input to a supervised learning model. The curves could be daily or hourly observations of a time varying metric, measured from $t=0$ up to $t=T$. Each individual curve $X_i(t)$ corresponds to a single example in the training data set. How do you encode the curves into a feature or set of features for your model?

The different curves would look something like:

enter image description here

The example mentioned was yield curves, but it was indicated that the question was regarding any set of time varying curves, as long as they were all over the same length of time $T$. Presumably the rate at which the curves are sampled is regular, but high enough that using the discrete samples as an input vector won't work.

The purpose of the supervised learning model wasn't specific: It could be forecasting, or regression, or some sort of logistic regression. The main idea was how would you feature engineer the curve data,

My answer was to either:

  • Choose a sufficiently high order n of polynomial, such that any one of the $X_i(t)$ can be reasonably approximated by a polynomial $\hat{X}_i = x_{i,0} + x_{i,1} t + x_{i,2} t^2 + \cdots + x_{i,n} t^n$ and then use $x_{i,0}, x_{i,1}, x_{i,2}, \ldots ,x_{i,n}$ as my $n+1$ features.

  • Or, treat each curve as a time series, and use the values of the ACF,PACF, the seasonality index, and the slope of the trend as inputs to our model.

The response I got was: neither, you need to use principal component analysis. But no further explanation was given.

My questions:

What is wrong with the approaches I suggested?

And how exactly can principal component analysis turn our curves into features? Wouldn't PCA just rotate our curves for us?

$\endgroup$
6
  • 3
    $\begingroup$ I'm voting to close this question, not because I think this is a bad question, but because I'm afraid you'll have to ask this to whomever made the test. For what it's worth, your second approach sounds much more reasonable than "you need to use PCA", and I personally cannot think of a simple reason why and how PCA would be suitable for this approach. $\endgroup$ May 30, 2019 at 7:09
  • 2
    $\begingroup$ Let me add that I don't think the question (not yours, the one you quote) is clear: First it states you already have curves, then it states these could be observations on an interval. A curve can be fit on a series of observations, but a series of observations is not a curve. $\endgroup$ May 30, 2019 at 7:18
  • 2
    $\begingroup$ Imagine a study of subjects that occasionally--but very rarely--exhibit a behavior of interest. Instruments on each subject record these behaviors for each second of a day by logging a zero when the behavior does not occur and a one when it does occur. If the behaviors are never simultaneous, these "curves" will be orthogonal and PCA won't accomplish more than counting the behaviors. The polynomial will be useless. This example shows that the nature of the data as well as the purpose of the analysis are key pieces of information needed to formulate a good answer. $\endgroup$
    – whuber
    May 30, 2019 at 17:06
  • $\begingroup$ @whuber I hope I have added sufficient detail on the nature of the data. The purpose of the exercise wasn't specified, although using the curves as casual inputs to a forecasting model was given as an example. $\endgroup$ May 30, 2019 at 19:20
  • $\begingroup$ It is standard to preprocess with PCA in supervised learning. I would be shocked if this wasn't mentioned a lot during the course of the semester. It is sort of "what you do" without really even thinking to help your model learn, and this question may even have been a "were you paying attention" type of question. Your given answers act as if the goal is to model the behavior of the curves, rather than simplify them and feed them to a supervised learning model. My guess is the prof didn't want to take the time to explain because this is something you should know already. $\endgroup$
    – neuronet
    Jun 11, 2019 at 2:51

1 Answer 1

6
$\begingroup$

The response was idiotic, my condolences. What you suggested is not unreasonable. It will probably not work, but how would you know in advance? Both your options were good starting points.

Although the simple polynomial approach is probably not a good idea in a literal sense, but in principle, it's not much different from Nelson-Siegel curves, e.g. see widely used Diebold Li approach. Nelson-Siegel parametrization is based on orthogonal Laguerre polynomials. Hence, your answer was not off the mark, in my opinion.

Treating yield curves as time series is also not wrong in principle. There's a bunch of paper where people do something along this line, e.g. this arxiv paper. In fact they compare performance vs. PCA.

The PCA is probably the best approach if you want to reduce number of inputs. You get first three component scores, and use them as your inputs instead of the original yield curve. First three PC usually cover more than 90% of variance. However, I wouldn't say that it's universally best approach. For instance, if you're modeling markets some entities may specifically be sensitive to 10 year US treasury rate, then maybe it's good to have it as a distinct input etc.

$\endgroup$
4
  • $\begingroup$ Thanks. I tried to get the paper, but it's behind a pay wall. My understanding it that PCA is a dimensionality transforming and/or reducing process. The thing I haven't wrapped my head around w/r to PCA is what is the dimensionality that we are trying to reduce in this case? We either have only 2 dimensions (in which case PCA won't really help) or an infinite dimension (since we are considering the curves to continuous). $\endgroup$ May 31, 2019 at 19:15
  • $\begingroup$ @Akaike'sChildren when you deal with yield curves (or interest rates in general) it is never truly curves in a continuous sense. It's convenient to represent them as such for mathematical treatment, but reality is that the curves are usually sets of rates of discrete tenors. For instance a swap "curve" can be 1, 2, 3, 5, 10 year tenors, i.e. spot rates maturing in these times in future. They could also be the same tenor, such as 10 years, but for a cross section of markets such as US Govt, German Bund, British whatever they call it etc. Hence, dimensionality is reduced from finite $\endgroup$
    – Aksakal
    May 31, 2019 at 19:43
  • $\begingroup$ "tenors" -> tensors? $\endgroup$ May 31, 2019 at 22:26
  • 1
    $\begingroup$ Tenor is a point on yield curve, a maturity of zero coupon bond and such $\endgroup$
    – Aksakal
    May 31, 2019 at 23:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.