Assume we have a Bayesian classifier with the three following features to determine whether a software user is a student, an admin, or an infiltrator (3 classifications).

LoginFrequency = { Never, Daily, Weekly }

LoginLocation = { Home, InCity, InState InCountry, OutOfCountry }

LoginDuration = { Seconds, Minutes, Hours, Days, Weeks }

  1. Assuming the features aren't independent, how many parameters need to be estimated, in total, to classify amongst the 3 types of users?

  2. What about if we assume the features are independent?


I'm having problems understanding how to count the number of estimated parameters in either case.

My guess for (2) is ${\displaystyle \prod_{i=1}^{3} X_i}$, where each $X_i$ is the number of values per feature, giving us $ 3 \times 5 \times 5 = 75$, which seems pretty wrong. As for (1), I don't know where to begin.


I'm only 2 weeks into a graduate ML course as an undergrad, and we've missed a lecture due to a snow storm so, as you might assume, I'm still much of a beginner. Hence, any help would be greatly appreciated.

  • $\begingroup$ Please see stats.stackexchange.com/tags/self-study/info . Here it would be useful to explain the reasoning behind the guess and/or what do you understand a Bayesian classifier to be and what kind of parameters it would have. $\endgroup$ – Juho Kokkala Feb 6 at 7:27

By the definition, events $A$ and $B$ are independent if

$$ p(A, B) = p(A) \, p(B) $$

For your calculations you need to be dealing with joint distributions. Joint distribution of two categorical variables is the "worst case scenario" in terms of parameters, since it's the number of combinations of all the levels, of all the variables. Assuming independence, this is significantly decreased, what follows from the definition.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.