I understand that you typically use three different data sets (train/validation/test) to acquire an unbiased estimate of the performance measurement, because the models are tuned to fit for the train dataset (for parameter learning) and the validation dataset (for hyperparameter learning).
But, if my goal is to simply compare different models rather than acquiring unbiased estimate of the performance measurement, do I still need train/validation/test split? Wouldn't train/test split be enough?
Hyperparameters are integral part of a model. In case you want a fair comparison between two models, this includes choosing the hyperparameter values. If you use only train and test sets you can select the hyperparameter values which optimize the performance over the train set, but this will likely result in a model with poor generalization abilities, thus failing to perform well on the test set. Unless you propose a structured way to choose hyperparameter values which will still preserve the model generalization abilities - you need a validation set.
