I am trying to understand few basic concepts of data mining, machine learning, etc. and I am new to this field

Say that I have a data sample and I have done maximum likelihood estimation or some other technique and found out good estimators for the parameters and figured out the data distribution that this particular sample belongs to.
My question is how is this knowledge useful in developing a model. Where exactly( I mean while using some machine learning algorithm to develop a good model) do I use the fact that I know the data belongs to this particular distribution?

(Sorry if I have wrongly written any technical jargon.)

  • $\begingroup$ I see strong parallels between this question and analogous statements in physics, such as "how do I use the fact that Newton posited that gravitational force follows an inverse square law?" $\endgroup$ – whuber Apr 13 at 22:43

Adding to Tim's answer, this is more of a comment than a full answer, but I disagree with the premise that

it is very important to know the underlying data distribution to develop a good model

You say you have noticed this

...in many textbooks

Can you provide some references to textbooks that make this assertion ?

Having said that, let's not forget that machine learning is very commonly used for prediction tasks, and the models, as Tim notes, are quite often black boxes and they literally do not care what the underlying data generation process / distribution(s) are.

On the other hand, models are also often used for inference, rather than prediction. It is very common to form a hypothesis about what the underlying data generation process is, then to collect some data and test the hypothesis that the model (an abstraction of the real underlying data generation process / distribution) fits the data well. This doesn't prove that the hypothesis is correct. At the end of the day, never forget that all models are wrong, but some are useful.

  • $\begingroup$ Thanks. I don't remember exactly where I have read it I will get back to you once I find the proper source(temporarily removed from the question). But this quora.com/… . led to my question. $\endgroup$ – Vyas Anirudh Apr 13 at 19:12

First of all, you never "know" the distribution. You make assumptions about distributions. If people say things like "height follows normal distribution" they do not mean that there's some force in the universe that makes the height be exactly consistent with some made up mathematical function. They mean that they've chosen normal distribution as an approximation of the real distribution of height. Moreover, if you knew the distribution, then you'd be using Bayes classifier, "the best you can get", and wouldn't even need to use the data, you'd just make decisions based on the known distribution.

If you use black-box machine learning models (say, random forest) then you don't really care about defining the distributions in mathematical terms. It makes a difference when you use statistical/machine learning models that are destined in terms of some distributions, i.e. they assume that your data follows some distribution, so if the assumption is not met, you don't have the guarantees that the authors gave for optimality of the results.

  • $\begingroup$ Thanks. Could you elaborate a little on the point your mentioned, " It makes a difference when you use statistical/machine learning models" $\endgroup$ – Vyas Anirudh Apr 13 at 19:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.