I have a series of data to which I want to fit my model. The model predicts the probability of success at a given value of x. I have a single data point at a number of points in this space. As I have a single point which is either pass or fail with a certain probability I believe I should fit using a Bernoulli maximum likelihood fit, is this correct? So i have a likelihood function which looks something like $$L(\theta,x) = \Pi^{n}_{i} \theta^{x_i}(1-\theta)^{1-x_i}$$ where n are my data points? This is just different enough to the usual Binomial likelihood case to have completely thrown me.
$\begingroup$
$\endgroup$
3
-
1$\begingroup$ Isn't $\prod_i^n\theta^{x_i}(1-\theta)^{1-x_i}$ the same as $\theta^{\sum_i x_i}(1-\theta)^{n-\sum_i x_i} = \theta^y(1-\theta)^{n-y}$ where $y = \sum_i x_i$ is a binomial $(n,\theta)$ random variable? $\endgroup$– Dilip SarwateDec 21, 2011 at 17:02
-
$\begingroup$ Wouldn't $y$ have the distribution ${n \choose k} \theta^y (1 - \theta)^{n-y}$ if $y = \sum_{i=1}^n x_i$ $\endgroup$– Sacha EpskampDec 21, 2011 at 17:53
-
$\begingroup$ @SachaEpskamp What is the relationship between $y$ and $k$? In any case, the probability mass function of $y$ is irrelevant. The likelihood of the observation or data $$x = (x_1,x_2,\ldots,x_n)$$ is exactly what Bowler stated it to be, and it is essentially a binomial likelihood. The extra constant factor $\binom{n}{y}$ (remember that $y$ is a function of the data $x$ and is thus fixed) that you have included makes no difference: the value of $\theta$ that maximizes $\theta^y(1-\theta)^{n-y}$ is the same as that which maximizes $\binom{n}{y}\theta^y(1-\theta)^{n-y}$. $\endgroup$– Dilip SarwateDec 21, 2011 at 22:12
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
1
If $x$ is a vector with independent measurements of a single variable which can be pass or fail, then this is the likelihood indeed, except that I think $L(\boldsymbol{x} \mid \theta)$ would be the proper notation.
EDIT: the binomial distribution is based on the sumscore. It would be the proper distribution if you were using $y = \sum_{i=1}^n x_i$.
-
$\begingroup$ Thining about it I asked a yes/no quesiton so this does actually answer it. Thanks. $\endgroup$– BowlerJan 16, 2012 at 15:28
$\begingroup$
$\endgroup$
1
I think what you want is logistic regression (or at least the likelihood that is maximized in logistic regression). If the wiki isn't enough, search for MLE fitting of logistic regression on Google.
-
2$\begingroup$ Thanks, logistic regression looks interesting its definitely a possibility. $\endgroup$– BowlerJan 8, 2012 at 15:47