Finding the best-fitting parametric distribution for empirical data I have an empirical dataset of samples that I'm trying to fit a (continuous) parametric distribution to from a list of possible distributions. I'm not trying to fit the parameters of a specific distribution.
Is there a "smart" (statistically speaking) method to do this? I've thought of the following:

*

*For each distribution, find the best parameters using for example MSE.

*For each distribution + best fitted parameters use Kolmogorov–Smirnov test or  Pearson's chi-squared test and choose the one with the highest score.

*For each distribution + best fitted parameters use Pearson's chi-squared test and choose the one with the highest score.

Which of the above would be the best approach? In 1, why not use the MSE as a criterion? Why do we need these tests?
Thanks.
 A: 
I have a set of samples that I'm trying to fit a (continuous) distribution to from a list of possible distributions.

The main question you need to deal with is why do that? 
What are you ultimately trying to achieve? What does this do for you that you can't get some other way? Why would your data come from some such list?

Is there a "smart" (statistically speaking) method to do this?

Yes: In general, don't. It's going to result in overfitting. 
However, if you must do this, you should go to some length to avoid assessing the suitability of the distributional model on the same set (or subset) of data you use to select between the models. (It's also very important you not use that same data for inference involving parameters.)
Sample splitting/cross validation will get you a long way.

For each distribution, find the best parameters using for example MSE.

Why would MSE be a good choice?

For each distribution + best fitted parameters use Kolmogorov–Smirnov test 

1) K-S assumes a specified, not fitted distribution. 
2) you're presumably comparing distributions with different numbers of parameters; why should they be comparable (models with more parameters should give a better fit -- if this is your criterion you'll simply choose a model with lots of parameters)
3) a test answers the wrong question (the question is whether the model is reasonable for some as-yet-unstated purpose, not whether some hypothesis test rejects or fails to reject)

Does that make any sense? 

I think it's ill-advised, but there's not presently enough information to provide a lot of guidance toward something better.

In 2, why not use the MSE as criterion? 

First, what are you calculating MSE of? Why is the MSE a good measure of your loss-function? 

Why do we need these tests for?

It's not clear what you're asking here. I don't think they serve any useful purpose for your kind of problem. Why would you use them for this?
A: The problem with K-S test is that it won't work out of the box in cases where you are estimating the theoretical distribution parameters from the sample, see item #3 here. There is a way to overcome this issue through Monte Carlo simulations, see here how to do it.
You can use $\chi^2$ goodness-of-fit test. It has its own issues, such as dependence on bins.
I don't think any of these tests would be good to automatically select the best fit. I would rather run the tests to reject the distributions that don't fit well, then deal with those which are left to pick one of them manually based on the knowledge of the underlying phenomenon.
A: My short answer, research how the data is being created. In other words, find or reasonablly justified a plausible generating model.
Next, find what distributions are associated with such generating process. For example, if the data follows a percentage change error model, consider a distribution like the Log-normal distribution (and not the classic assumption of a Normal probability model).
This approach is more likely to produce better (and supportable) results.
