My question is more about the methodology. Assuming in some experiment we have measured quantity $y$ per each unit of time $x.$ So $y$ and $x$ form our data set here. Moreover, we know that they are related by a power law type of relation, e.g., $y = D x^{\alpha},$ where $D$ is just a constant.

Now to extract $\alpha$ from the data-set, I know two ways:

  • a) Calculating the logs of our data, we can then compute the derivative of the $\ln(y)$ w.r.t $\ln(x),$ so $\frac{d\ln y}{d\ln x}=\alpha$ and extract the power. One problem with this is that the sampling may not have been done logarithmically, so the spacings between the log'ed values are different. That means numerically it is going to be hard to accurately compute such derivative.

  • b) Another way would be: taking the logs again as in above, but then we just fit the log'ed data with a line, the slope of which should give us an average $\alpha,$ right? Assuming this is correct, one problem is that if $\alpha$ is changing during different time scales of the experiment, the above fit wouldn't capture it. Maybe one could perform the fits piece-wise.


  • Have I laid out the above methods correctly? (e.g., is b) correct?)
  • Does one method come more recommended or it really depends on the context? (i.e., in view of the aforementioned difficulties) Finally, please feel free to suggest other ways of extracting $\alpha$ if you know of different methods, I'm very curious to find out.

(If you prefer explaining your method with an example, I have created dummy data for purposes of illustration, here's the link, first column is $x$ and second column $y.$)

Clarifications upon reading discussions in the comment section:

The aim here is only to tackle the problem of how to reasonably estimate (for instance by fitting) power laws that describe a given bivariate data set, and more precisely, finding power laws that correspond to each region of interest [*] (i.e., subsets of the data). With this in mind, what the user Nick Cox has proposed as answer, is precisely on point.

[*]: thus e.g., fitting subsets of the data, and more contextually, looking for different power laws at different time-scales, because for instance from a physical point of view we expect the data to exhibit different power laws.

  • 3
    $\begingroup$ @NickCox Dear Nick, very well captured, this is indeed what I am trying to learn to do (in terms of applications, e.g., in context of physics, this type of analysis is relevant when extracting different regime in the dynamics of a system, by measuring mean-squared-displacement of particles as function of time, and seeing what power-law governs the MSD during which time scales). Sure, I will generate dummy data that can be utilized to illustrate here. $\endgroup$ – user929304 Mar 1 '18 at 15:06
  • 3
    $\begingroup$ @DJohnson Sorry, but I think you misread the question in your enthusiasm to link to one of your top interests. The OP has confirmed that. You might as well say that all questions in which logarithms are mentioned or implied are the same question. Gumbel was there just copying from Russell, but failing to cite him, incidentally $\endgroup$ – Nick Cox Mar 2 '18 at 13:00
  • 3
    $\begingroup$ @DJohnson There is no sense whatsoever in which e.g. the analysis in my answer is an extension of the theory for univariate distributions you point to. I think you're just confusing an otherwise well-focused thread: the first paragraph of the original question is crystal clear on what the problem is, and that is reinforced by the OP's comment on the other answer to underline the distinction. I am happy to let readers decide on whatever they find here that is focused, interesting and helpful. $\endgroup$ – Nick Cox Mar 3 '18 at 16:03
  • 3
    $\begingroup$ @DJohnson We don't expect to convince each other, but I do care that other people can be misled by error, confusion and irrelevance. These comments aren't pertinent or even factual: power functions aren't nonlinear exponential models; they go back at least to the 18th century; nor are they univariate processes, whatever that means. Marginal and even conditional distributions don't need to be considered to apply the analysis in my answer. $\endgroup$ – Nick Cox Mar 4 '18 at 13:21
  • 3
    $\begingroup$ Power functions $y = ax^b$ as are discussed in this thread (except in your comments) are not S-shaped. They are convex or concave curves in $(x, y)$ space and straight lines in $(\log x, \log y)$ space. $\endgroup$ – Nick Cox Mar 5 '18 at 12:27

As in my first comment on the question I see this as being entirely about power laws for bivariate data. (The inclination to read it otherwise is puzzling.)

Based on the posted data, I did local polynomial smoothing; the choices here are no more than not very smart defaults in the program used, but equally there doesn't seem much need to play with other choices. (The $R^2$ here is just the square of the correlation between observed and smoothed; 1 isn't even a target as interpolating the data could achieve that.)

enter image description here

It seems clear that the slope stabilises fairly quickly and systematically in logarithmic space, so that numerical differentiation could give you estimates of slope as it changes.

  • $\begingroup$ Thank you for performing the test, I don't know what fit function you have used (or what program), but I have tried taking the derivative of this sample data (in log space) in python and it's fluctuating all over the place so to speak... it may be my method in python is not appropriate (I've simply used np.gradient(np.log(y))/np.gradient(np.log(x)) np standing for numpy here. It would be interesting to see how the derivative turns out with your program if you could kindly check. $\endgroup$ – user929304 Mar 1 '18 at 15:35
  • $\begingroup$ As said, this is local polynomial smoothing; there is no global function, just linear regression within moving windows and with weights within the window. I imagine that, as I should have emphasised, you have to smooth first because numeric differentiation is sensitive to local noise. I'm calling up lpoly in Stata documented at stata.com/manuals/rlpoly.pdf except that I am calling it through my own program which just chooses different defaults. Conversely, sorry, but I don't use python and can't advise on how to do it. Presumably, there will be implementations to choose from. $\endgroup$ – Nick Cox Mar 1 '18 at 15:45
  • $\begingroup$ Alright, fair enough. If you find the time to perform the derivative in your program, let me know please how it turns out, it would be a sanity check for me that I'm doing things correctly in python. $\endgroup$ – user929304 Mar 1 '18 at 15:48
  • $\begingroup$ I get final slopes about 0.3. $\endgroup$ – Nick Cox Mar 1 '18 at 15:52

Edit: The question is actually about data in which $x$ and $y$ are allowed to vary. This answer is not relevant to that case, but only to the case when $x$ is a quantity and you want to fit a distribution of the form $f(x) \propto x^{\alpha}$ to it. However, I am leaving it up because it links to a paper which is state-of-the-art for this case, which may be useful for some readers who are confused (like I was) by the question title.

I suggest reading the following paper by Clauset, Shalizi and Newman. It recommends using maximum likelihood to fit a power law (so, not taking logs and fitting a straight line at all).

Even if you know that the data follow a power law and you are just trying to find an exponent, it might still not be appropriate to fit a straight line to the log-log plot with least squares. This is because, if you fit such a line, any statistical calculations are implicitly assuming that the errors in measuring $\log(y)$ are normally distributed, which may not be true, depending on circumstances. See Appendix A of the above paper.

  • 1
    $\begingroup$ Very interesting! thanks a lot for bringing this paper to my attention. I wonder, for Appendix A, isn't the case being discussed only of relevant for when the lhs is a probability distribution? $\endgroup$ – user929304 Mar 1 '18 at 15:37
  • 7
    $\begingroup$ This is a very good paper, & always worth a read. However, note that it pertains to a univariate power law distribution, not a bivariate power law relationship. This is the point @NickCox was trying to bring out. $\endgroup$ – gung - Reinstate Monica Mar 5 '18 at 14:17
  • $\begingroup$ Yes, I understand the point but was not sure whether to delete my answer, since the OP seemed to find it useful. $\endgroup$ – Flounderer Mar 5 '18 at 15:01
  • 4
    $\begingroup$ I'd say: keep your answer. First, the title of the thread may make it entirely relevant to some readers. Second, it should now be clear to careful readers of the thread what is what. $\endgroup$ – Nick Cox Mar 5 '18 at 15:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.