The tag has no wiki summary.

learn more… | top users | synonyms

1
vote
0answers
8 views

Estimating distribution of traffic across all Web pages

The web is estimated to have approx 5 billion web pages ( http://www.worldwidewebsize.com/ ). How would you go about estimating a CDF to know how many pages are required that make up, say, 90% of ...
0
votes
0answers
41 views

Connection between power law and Zipf's law

I am trying to better understand the connection between the power law distribution and Zipf's distribution (law). There is a neat explanation in [1]. The article suggests that as we can derivate the ...
1
vote
0answers
95 views

Using Zipf's law to select top K ngrams?

Lets say we extract all the ngrams (unigrams, bigrams, trigrams) along with their frequency from a text. Now if we want to select say some Top K ngrams, is there a way to estimate a good value of K ...
3
votes
0answers
52 views

Zipf's vs Self-similar: are they really the same

Recently I ran into a test using both zipf's and self-similar generated datasets. I followed the description from Jim Gray's paper on generating such datasets (Quickly Generating Billon-Record ...
1
vote
0answers
129 views

Sales prediction of Amazon Sales Rank

I understand that Zipf's Rank-size distribution would allow me to estimate the sales of a book given the amazon sales rank. How can I fit the distribution without knowing book sales leading to a rank. ...
0
votes
1answer
402 views

How to calculate zipf exponent of page degree distribution?

I want to calculate the exponent of the function which shows zipf's law in data. I'm working in the context of graph node degree analysis and want to comprehend results shown by Border et al. on a new ...
0
votes
1answer
239 views

Discretize frequency of words that follow zipfian distribution

How could I discretize the frequency of words found in a corpus that follow a zipfian distribution? Are there standard methods? It should create bins of exponential-increasing size. My goal is to ...
1
vote
0answers
318 views

Mean and median in Zipf's distribution

I have a collection of $10^5$ essays, each of which on average contains $10^3$ distinct words. There are $10^6$ distinct words in the entire collection. If I index every word what is the mean and ...
7
votes
3answers
2k views

How to estimate parameters for Zipf truncated distribution from a data sample?

I have a problem with the estimation parameter for Zipf. My situation is the following: I have a sample set (measured from an experiment that generates calls that should follow a Zipf distribution). ...
12
votes
2answers
4k views

How to calculate Zipf's law coefficient from a set of top frequencies?

I have several query frequencies, and I need to estimate the coefficient of Zipf's law. These are the top frequencies: ...
6
votes
1answer
966 views

Is principal components analysis valid if the distribution(s) are Zipf like? What would be similar to PCA but suited to non gaussian data?

I'm analyzing people based on their twitter stream. We are using a 'word bag' model of users, which basically amounts to counting how often each word appears in a persons twitter stream (and then ...