0
$\begingroup$

I have yet to choose a classifier to get my model but I wanted to see which possible classifiers fit what I'm looking for. In fact, I need an algorithm to establish offline training this way:

  • The first training starts after a month of collecting data (I know it's a short time but I want to see how it performs and how it gets better with time and more data) => first model is made
  • After another month, I've got data collected after two months in total, so I want to establish a new model since my training data has just doubled and the model should provide better results => second model
  • this continues on and on for a year...

Question1: After two months, when I want to add the new training data, would the previous model be updated to make the new model or would the training start all over as if I don't have a previous model?

Question2: Do all classifiers behave the same when new training samples are introduced?

I don't know if this is relevant, but my problem is a low-dimensional one: about 5 inputs (user location, date, time...) to predict one output.

$\endgroup$
0
$\begingroup$

The answer to question 1 is that it depends on the model/optimization method used. In principle, it takes a specialized approach to use new data in order to update the model without retraining (with the entire dataset). For more on this see: https://en.wikipedia.org/wiki/Online_machine_learning

The answer to question 2 is no. As a simplistic (and hand wavy) example, consider the case where you observe a sequence of observations: $$ (y_1, X_1),...(y_n,X_n) \in \mathbb{R^p}\times\mathbb{R}, $$ and that you are interested in estimating a linear regression: $$ y_i = X_i\beta + \varepsilon_i. $$ One possibility is to do so via the usual least squares estimator: $$ \hat\beta = (X^{T}X)^{-1}X^{T}y. $$

Another option, in case outliers are of concern, is to use a robust estimating equation: $$ \hat\beta = \arg\max_\beta \varphi(X\beta - y), $$ for more on robust regression see- https://en.wikipedia.org/wiki/Robust_regression. Now, suppose that the $n+1$ observation is an outlier, so for example, $$ y_{n+1} = 10^5 * \max_{i\in 1:n} |x_i|. $$

In this case the non-robust estimate will likely change by a lot and the robust regression estimate will not be effected.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.