1
$\begingroup$

I want to study orthographical variants, for example:

Can firefighter, fire-fighter and fire fighter be considered orthographical variants (the difference in frequency in a corpus is not statistically/practically significant), or should one or more of these orthographical forms be considered "wrong"/"irregular" (the difference in frequency is statistically/practically significant)?

In a corpus of 926,766,504 words, I get the following frequency counts (on lemmas):

  • firefighter = 3,349
  • fire-fighter = 336
  • fire fighter = 1,338

What statistical measure(s) can I use to say, for example, that there is not a large enough difference between firefighter and fire fighter to consider the latter "irregular", but that the difference between firefighter and fire-fighter is significant?

One specific sub-question:

If I work with probabilities (e.g. firefighter = 0,667; fire-fighter = 0,067; fire fighter = 0,266), is there a way to measure if the difference is significant? (Beyond just stating the obvious that the one occurs with a higher probability than the other.)

Improve this question
$\endgroup$
3
  • $\begingroup$ A corpus cannot tell you if an orthographic variant is correct or not. Typically a panel of native speakers is used to rate correctness. $\endgroup$
    – Anonymous Physicist
    Nov 28, 2016 at 8:23
  • $\begingroup$ I used "wrong" specifically between inverted commas, because that is not the real issue here. It is more about whether a certain form occurs so frequently that it could be considered a viable spelling alternative. Such statistical information would typically inform a panel of experts, as you mentioned. $\endgroup$
    – Gerhard Van Huyssteen
    Nov 28, 2016 at 14:27
  • $\begingroup$ What is the difference between "correct" and "viable spelling alternative?" I do not think there is one. I disagree with you about the "real issue." The real issue is that a corpus does not contain correctness information. $\endgroup$
    – Anonymous Physicist
    Nov 28, 2016 at 23:21

1 Answer 1

Reset to default
1
$\begingroup$

This might answer your question: Statistical significance between letter frequencies in different corpora

You can use a chi square test of independence.

Improve this answer
$\endgroup$
1
  • $\begingroup$ As I understand it, this is specifically not an option, because I draw frequencies from the same corpus. Hence, the assumption of independence is violated. Please correct me if I'm wrong. (I'm a statistics virgin!) $\endgroup$
    – Gerhard Van Huyssteen
    Nov 28, 2016 at 14:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.