0
$\begingroup$

I am trying to process a set of tweets, by using its length to determine the probability of spam.

The data looks like this, given 2000 tweets, n is the tweet length (number of words)

when n = 1, relevant tweets = 0
when n = 2, relevant tweets = 10
when n = 3, relevant tweets = 20
when n = 4, relevant tweets = 100
when n = 5, relevant tweets = 200,
...
...
when n = 9, relevant tweets = 10
when n = 10, relevant tweets = 0.

My question is, how do I build a normal distribution based on this set of data, and calculate the probability of a tweet being spam, given n?

Thanks very much. This is of my own research work and it is not homework. Sorry if the questions are not asked in proper statistical way as I have limited knowledge on probability and normal distribution.

Improve this question
$\endgroup$
4
  • $\begingroup$ by "relevant tweets" do you mean the tweets of that length that were spam? $\endgroup$
    – Chan-Ho Suh
    Commented Nov 8, 2013 at 3:59
  • $\begingroup$ I agree with David's response. Your data is not normal because it is discrete. If you want to blindly apply a normal curve, calculate the mean and variance and plot a normal curve over the histogram of your dataset. $\endgroup$
    – Drew75
    Commented Nov 8, 2013 at 6:37
  • $\begingroup$ Thanks for your input guys, if that's the case, do I have to use something like a binomial distribution or something else? $\endgroup$
    – drhanlau
    Commented Nov 8, 2013 at 8:09
  • $\begingroup$ What are you thinking of using the normal/binomial/whatever distribution for? A hypothesis test? $\endgroup$
    – David Marx
    Commented Nov 9, 2013 at 16:35

1 Answer 1

Reset to default
3
$\begingroup$

The probability of a tweet being spam given n is just the proportion of tweets for a given n that are spam. If for n=4 you had 10 spam tweets out of the 100 tweets observed for that level of n, then P(tweet is spam|n=4) = 10/100 = 0.1 = 10%.

It's not clear to me what you mean by "build a normal distribution based on this data set."

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.