23
$\begingroup$

I would like to use a binary logistic regression model in the context of streaming data (multidimensional time series) in order to predict the value of the dependent variable of the data (i.e. row) that just arrived, given the past observations. As far as I know, logistic regression is traditionally used for postmortem analysis, where each dependent variable has already been set (either by inspection, or by the nature of the study).

What happens in the case of time series though, where we want to make prediction (on the fly) about the dependent variable in terms of historical data (for example in a time window of the last $t$ seconds) and, of course, the previous estimates of the dependent variable?

And if you see the above system over time, how it should be constructed in order for the regression to work? Do we have to train it first by labeling, let's say, the first 50 rows of our data (i.e. setting the dependent variable to 0 or 1) and then use the current estimate of vector ${\beta}$ to estimate the new probability of the dependent variable being 0 or 1 for the data that just arrived (i.e. the new row that was just added to the system)?

To make my problem more clear, I am trying to build a system that parses a dataset row by row and tries to make prediction of a binary outcome (dependent variable) , given the knowledge (observation or estimation) of all the previous dependent or explanatory variables that have arrived in a fixed time window. My system is in Rerl and uses R for the inference.

Improve this question
$\endgroup$
9
  • 6
    $\begingroup$ can you assume a correlation structure on your data? Your case is a special case of GLMM with logit link, but the correlation structure in the time series data must be modeled correctly to get a reasonable answer. $\endgroup$
    – suncoolsu
    Commented Jul 18, 2011 at 5:24
  • 1
    $\begingroup$ when you say time series, $y_t$ would have some kind of relation to $y_{t-1}$. Or can it be assumed to be independent? $\endgroup$
    – suncoolsu
    Commented Jul 18, 2011 at 5:39
  • 2
    $\begingroup$ could you please give a concise description of your data for me to give a concrete solution? you problem can be solved something like this stat.ethz.ch/pipermail/r-sig-mixed-models/2010q4/004530.html $\endgroup$
    – suncoolsu
    Commented Jul 18, 2011 at 8:42
  • 2
    $\begingroup$ I have a network traffic time series of the following form: Protocol,SrcIP SrcPort,DestIP,DestPort,TimeSec,Timeusec,PackLength TCP,200.80.199.105,3523,207.216.233.144,9658,11223344,941818,62 UDP,142.144.155.120,1751,244.72.151.2,1935, 11223344,941843,60 I want to estimate if a packet (or group of packets) is malicious by using knowledge from labeled datasets to build a self-trained model. The averaging I was talking about is applied at the above metrics in order to give a level of aggregation and make the system more practical for high volume traffic. $\endgroup$
    – Regressor
    Commented Jul 19, 2011 at 2:29
  • 2
    $\begingroup$ This really sounds like a job for a support vector machine. Am I missing something? If you're really concerned about autocorrelation or the time-series structure of your data, you might try ARIMA and/or a multilevel longitudinal model. On longitudinal models, I recommend Willet and Singer's Applied Longitudinal Data Analysis, for which the UCLA ATS site has R code examples. $\endgroup$
    – ashaw
    Commented Sep 30, 2011 at 15:34

1 Answer 1

Reset to default
8
$\begingroup$

There are two methods to consider:

  1. Only use the last $\mathrm{N}$ input samples. Assuming your input signal is of dimension $\mathrm{D}$, then you have $\mathrm{N} \times \mathrm{D}$ samples per ground truth label. This way you can train using any classifier you like, including logistic regression. This way, each output is considered independent from all other outputs.

  2. Use the last $\mathrm{N}$ input samples and the last $\mathrm{N}$ outputs you have generated. The problem is then similar to Viterbi decoding. You could generate a non-binary score based on the input samples and combine the score of multiple samples using a Viterbi decoder. This is better than method 1. if you know something about the temporal relation between the outputs.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.