Should exploratory data analysis include validation set? I know that EDA should be performed on the training set but not on the test set. 
But my question is: we usually split the training set into training and validation datasets. Should we perform EDA on both of them?
 A: Your question is related to this one about EDA on train vs. test and this one on the purpose of validation and test sets.
Why have a validation set at all? A validation set allows you to explore and evaluate many models/hyperparameter settings. As you make decisions about which model to use, the validation set allows you to validate whether that decision was actually good, without overfitting to your test set. Similarly with EDA, if you make any decisions about your model (which features you are going to select, etc.), the validation set would help you validate whether your EDA decisions are good.
Therefore, I would perform EDA only on the training set and use the validation set to evaluate the quality of any decisions you made on your training set.
Edit
Cross-validation (CV) complicates this a little. The core principle is that the validation set should help you validate any decisions you make. Making decisions based on the validation set will inflate (or deflate, as appropriate) any model scores on the validation set. These inflated scores will be more representative of the training set scores and less representative of the test set scores, thus defeating the purpose of a validation set.
In k-fold CV, you could have k separate EDA phases, one performed on the train split of each fold. You would have to make decisions based solely on the results of each EDA without transferring decisions between each one. It would be difficult to remain unbiased in the later folds and impractical to analyze the data k times.
Perhaps a more practical option would be to first make train/validation/test splits. Then perform EDA on the train set and tune your model using cross-validation on the train set. The model's hyperparameters would still be tuned with the benefits of CV. The validation set would then allow you to evaluate simultaneously both the decisions made in your EDA and the tuning process. The validation scores would be more representative of the test scores and you would be less likely to overfit. You would need a dataset large enough to handle all of this splitting.
This is a very strict paradigm and I would be surprised to learn that researchers/professionals actually follow it. Somehow, you need to evaluate all your decisions empirically while balancing the fine-tuning of your model without overfitting to a particular subset of the data.
A: You are talking about two different sets of steps in your post.

*

*Data visualization, Exploratory Data Analysis


*Model training, evaluation and testing
In exploratory data analysis one analyzes the data sets to summarize their main characteristics, often with visual methods. So you should consider complete data set there. In case you split the data set into train, validate and test before EDA, you might be missing some important information in EDA. For example, you could miss the outliers because they are a part of test data.
After you are done with EDA, you need to keep the data set intact for data pre-processing and transformation as well. After that you can split the data set. If you split data set before pre-processing and transformation, you would be training your model on one type of data set and testing on something else. For example, let us say you are trying to predict if a person should be given a loan or not. There is an attribute for 'salary' and 'age' in the data set. Let us say as a part of pre-processing you decide to apply normalization. If you split data set before pre-processing, you would be training your model on normalized 'salary' and 'age' data whereas evaluating, testing the model on the original 'salary' and 'age' data.
Some people might want to do only EDA for insights and not go for model training and testing.
So, you should always split the data set just before you start model training.
A: My thinking is that it depends on what you are trying to accomplish. If your goal is to:

*

*Build a model that generalizes to unseen data:
Then you want to keep the training, validation, and testing set separate for
all stages of the pipeline. "Separate at the earliest possible opportunity"


*Understand the population:
Then EDA on the entire dataset is permissible.
