Timeline for Characterizing/Fitting Word Count Data into Zipf / Power Law / LogNormal
Current License: CC BY-SA 3.0
23 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Sep 3, 2023 at 23:01 | answer | added | jvdillon | timeline score: 0 | |
| Feb 26, 2019 at 20:50 | answer | added | ivangtorre | timeline score: 0 | |
| Mar 16, 2018 at 18:35 | history | edited | born to hula | CC BY-SA 3.0 |
deleted 747 characters in body
|
| S Mar 12, 2018 at 20:44 | history | bounty ended | CommunityBot | ||
| S Mar 12, 2018 at 20:44 | history | notice removed | CommunityBot | ||
| Mar 5, 2018 at 19:27 | history | edited | born to hula | CC BY-SA 3.0 |
edited title
|
| Mar 4, 2018 at 20:17 | answer | added | David Dale | timeline score: 6 | |
| Mar 4, 2018 at 19:28 | history | tweeted | twitter.com/StackStats/status/970380502187339776 | ||
| Mar 4, 2018 at 19:01 | comment | added | born to hula | @MarkWhite changed the scope and the question a bit. I'm no longer truncating the distribution - decided to use the raw data. Note that "the" is clearly the most frequent word. Also started a bounty for this btw. | |
| Mar 4, 2018 at 19:00 | history | edited | born to hula | CC BY-SA 3.0 |
Added analysis updates
|
| S Mar 4, 2018 at 18:54 | history | bounty started | born to hula | ||
| S Mar 4, 2018 at 18:54 | history | notice added | born to hula | Improve details | |
| Mar 2, 2018 at 21:54 | comment | added | Mark White | check out cran.r-project.org/web/packages/aster/vignettes/trunc.pdf. I would look for some type of Python implementation for truncated negative-binomial distributions | |
| Mar 2, 2018 at 19:13 | history | edited | born to hula | CC BY-SA 3.0 |
added 195 characters in body
|
| Mar 2, 2018 at 19:09 | comment | added | born to hula | You're correct @Mark White. I removed words with occurrence inferior to five, and also removed some stopwords such as "the", "of" etc. Also looks like Poisson or Negative Binomial to me - however mean is different from variance. This would leave me with Negative Binomial - I just wanted to know if there is some way to calculate the Goodness of Fit for it, or any other possible method to accept/reject the null hypothesis. | |
| Mar 2, 2018 at 18:11 | comment | added | Mark White | What is the minimum of your data? Did you truncate the distribution by ignoring anything with a count less than a certain value? To me, this looks like a Poisson or negative binomial distribution, truncated at 5 | |
| Mar 2, 2018 at 17:48 | history | edited | born to hula | CC BY-SA 3.0 |
added 227 characters in body; edited title
|
| Mar 2, 2018 at 17:45 | comment | added | born to hula | Thank you @www3. I was looking for a pythonic way to test if this data fits into a given distribution. I could inspect my plot against the plots of the distributions from the exponential family, but I wonder if there is another way to do this using scipy | |
| Mar 1, 2018 at 20:08 | comment | added | www3 | The distribution that maximizes the entropy given a bunch of sufficient statistics (like you have) is an exponential family with those sufficient statistics. For example here: web.stanford.edu/class/stats311/Lectures/lec-07.pdf | |
| Mar 1, 2018 at 18:56 | history | edited | born to hula | CC BY-SA 3.0 |
edited tags; edited title
|
| Mar 1, 2018 at 14:18 | history | edited | born to hula | CC BY-SA 3.0 |
edited title
|
| Mar 1, 2018 at 14:01 | review | First posts | |||
| Mar 1, 2018 at 14:42 | |||||
| Mar 1, 2018 at 14:00 | history | asked | born to hula | CC BY-SA 3.0 |