If you're trying to classify some input, $\mathbf{x} \in \mathbb{R}^{n}$, to one of $d$ classes using a model with parameters, $\theta$, how are you supposed to think about the experiment of learning parameters in a probability measure-theoretic sense?

I'm a novice, but in the machine learning examples I've read, each one would be treated as a random variable: $X$ for the input, $Y$ for the output, and perhaps $\theta$ for the parameters.

How do we think about the sample space, $\Omega$, for the entire "experiment"? Does each random variable have an underlying "experiment" where the sample space is the possible values in takes on and the random variable maps directly to the outcomes, $\omega$? And then perhaps we create a composed experiment with a sample space, $\Omega_{X} \times \Omega_{Y} \times \Omega_{\theta}$?

Any resources on how measure theory is connected to the basic problems of machine learning would be helpful.

  • 1
    $\begingroup$ such a fundamental question but no answers :/ $\endgroup$ – Erik Hambardzumyan Feb 4 at 18:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.