1
$\begingroup$

I want to calculate the exponent of the function which shows zipf's law in data. I'm working in the context of graph node degree analysis and want to comprehend results shown by Border et al. on a new dataset. I have pairs of values where the first value denotes the degree and the second one the number of nodes with that degree, like that:

1   188638380
2   20839145
3   9049892
4   5427599
[...]
641531  1
663136  1
820267  1
920489  1

I'm searching for a program that calculates the zipf exponent automatically and stumbled about zipfR of the R language. Unfortunately I'm unfamiliar with R and with statistic as well. Furthermore zipfR don't want to work with me. I needed to use degrees<-data-frame(scan("path"), list(0,0),skip=1))to get the data out of the file that's because I have about 13.000 data points. When I tried to use functions of zipfR I got errors that zipfR can't work with that.

Afterwards I tried to adapt the algorithm shown when the question about frequency's shows up but the resulting exponent is far to small. There might be some reasons for it I'm not sure of(I tend to reason 2):

  1. The code was written for single values and not for pairs. Did the algorithm assume iterative raising integers? Because in my dataset are gaps
  2. Broder might truncated values. How to estimate a good point for truncation?

To let you see what I've done:

dataset<-data.frame(scan("/data/dd_in.txt", list(0,0),skip=1))
p<-dataset/sum(dataset)
lzipf<-function(s,N) - s*log(1:N)-log(sum(1/(1:N)^s))
opt.f<-function(s) sum((log(p)-lzipf(s,920489))^2)
opt <- optimize(opt.f,c(0.5,4))

If I get it right, the values after c are the range for the exponent. I expect an value of around 2 for the exponent. The value in the opt.f function of 920.489 is the maximum value of the degrees. Is that correct? What about the number of pages with a certain degree? Where can I find them or add them? Is that implicitly done due to the p value or did I need to fill the gaps with 0?

$\endgroup$
1
$\begingroup$

I think i finally find out what to do.

data = read.delim("inputfile.txt", header=F)
lim = dim(data)[1];
degree = data[1:lim,1]
numberOfPages = data[1:lim,2]

p = numberOfPages / sum(numberOfPages)
fullDegree = c(1:max(degree))

lzipf<-function(s,seenValues) - s*log(seenValues)-log(sum(1/(seenValues)^s))
opt.f<-function(s) sum((log(p)-lzipf(s, degree))^2)
opt <- optimize(opt.f,c(0.5,4))

That solved my problem and gives me expected results.

Greetings, merando

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.