I am writing a report to show my results where, if there are more than 3 consecutive English character in a term, that term is noise. For example, "wooooh", "Noooo", etc. I have collected 100 sample and manually evaluated them. May I know what test should I perform to verify my hypothesis? Is it something like ANOVA ?

  • 3
    $\begingroup$ What hypothesis? What model? What is it you're actually trying to do, in plain, simple terms? $\endgroup$ – Glen_b Dec 13 '12 at 8:15
  • 1
    $\begingroup$ ...and how did you collect the samples? $\endgroup$ – conjugateprior Dec 13 '12 at 10:52

If your goal is to see if your method works, then you need a sample that includes both "noise" words and "regular" words.

First, you'd need to precisely define those terms. I don't know how you are defining them. "Noooo" is not noise in English text - it is often an extreme version of "No".

You'd have to then figure out how to sample. I'd say you would need some very large selection of random words of English. Do you want spoken or written English? From what type of source? If you include graphic novels and comics you'll get quite a lot of words that meet your definition. But, since noise words are rare, you would need a large selection - how large? I don't know. Thousands or tens of thousands of words seems like a good range, intuitively.

Then you'd have to decide which of these words is noise and which is not. Depending on your definition, this might be very hard.

Only then could you analyze your data. I'd say logistic regression would be a good starting point, where the dependent variable would be Noise v. not and the independent variable would be having 3 consecutive characters.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.