0
$\begingroup$

I am training an SVM model to predict the trend of stock prices (one-day ahead predictions. Classification task). It Had completely slipped from my mind that SVMs assume IID data until I had a conversation with a friend.

This made me rethink about my approach and I have a few questions.

  1. Why does SVM assume IID data in the first place?

  2. By IID, does it mean that all the features should be linearly uncorrelated or should there not be any non-linear relationships as well?

The features include basic stock data (Open, High, Low, Close, Volume) and technical indicators (derived from basic data). Of course, the technical indicators will have dependencies with basic data. So I removed the basic data from the feature space. However, this does mean that the technical indicators are not related. Also,

  1. How does one make sure that the data is identically distributed? Does scaling (Normalization or Standardization) help in this case?

Lastly, I have read some papers wherein people have used NN and SVM (both assume IID data) for stock trend prediction, but nowhere did I see any mention of the IID assumption and the fact that stock data is actually not uncorrelated and to some extent exhibits some autocorrelation as well. So,

  1. How does one justify the use of non IID data as input to such algorithms which assume that the data is IID?

Thank you :)

P.S.: I had posted this question on Data Science Stack Exchange but haven't received any response yet. Hence, I posted it here.

$\endgroup$

1 Answer 1

1
$\begingroup$
  1. Why does SVM assume IID data in the first place?

The problem is not specific to SVM per see, but is more general and already answered in the On the importance of the i.i.d. assumption in statistical learning thread, and here is an example applied to logistic regression.

  1. By IID, does it mean that all the features should be linearly uncorrelated or should there not be any non-linear relationships as well?

You can start with the What are i.i.d. random variables? and What is implied by i.i.d.? threads. Independence is a much broader assumption than being uncorrelated. Correlation is about linear relationship, independence about any relationship whatsoever.

  1. How does one make sure that the data is identically distributed? Does scaling (Normalization or Standardization) help in this case?

It is an assumption that you make. You assume that all your samples come from the same distribution, are of the "same kind". You also assume that they are independent of each other, so for example there is no temporal dependency (time-series), or clustering.

Scaling, standardization, etc. do not change the distributions. Try applying them to any data and plot histograms of before vs after, you'll notice that after scaling the scale of $x$-axis differs, but the shape of the histogram stays the same. They certainly do nothing to make the samples independent.

  1. How does one justify the use of non IID data as input to such algorithms which assume that the data is IID?

You'll find many answers in the threads linked above, but TL;DR is that it is just an assumption. First, no data is truly i.i.d., this is an abstract concept. The assumption being there tells you that if it is broken, you have no guarantees for optimality of the results that you would normally have. We often use statistical or machine learning methods knowingly that some of the assumptions are being broken, but there are cases where you do not have better alternatives. In such a case, just keep in mind that you are using the method with the safeties off, so you need to be extra careful about validating and interpreting the results.

$\endgroup$
1
  • $\begingroup$ Thank you @Tim for the thread links : ) $\endgroup$ May 12, 2021 at 10:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.