Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

A power-law is a function that increases proportionally to a power of its argument (ax^b). Often seen in fitted relationships or in densities (power-law distributions).

0
votes
0answers
10 views

Method for clustering multiple power law variables

Are there any clustering methods suitable for clustering high dimensional datasets that are composed of multiple highly skewed variables, including power law distributed variables? I am considering ...
0
votes
0answers
21 views

How well is a power law distribution described by the first four moments?

For a normal distribution, the first two moments (mean and variance) are sufficient statistics for the entire distribution. Suppose I have a power law distribution, and I have data on the first, ...
3
votes
0answers
35 views

Is the truncated power law a heavy-tailed distribution?

A heavy-tailed distribution is often defined as a distribution with a tail that is not exponentially bounded. A truncated power law (or power law with exponential cut-off) is a distribution that ...
0
votes
0answers
14 views

How to fit a distribution to a function of binned values

I am interested in fitting power law or Pareto distributions to a set of values. First, we assume that the abundance $A$ of an organism is related to mass $M$ with a power law with exponent -2: $A \...
3
votes
3answers
103 views

Most accurate way to express the income of the top 1% in a power distribution?

I have an array of 50,972 household incomes for a small metro area, and I want to measure how much the so-called top "one-percent" make (e.g., the 99th percentile). As you'd expect, it's a power curve:...
2
votes
0answers
46 views

Is KS test really appropriate when validating a power law/estimating power law parameters?

I'm attempting to find out whether some highly skewed data are drawn from a power law distribution, following the popular paper by Clauset, Shalizi and Newman, 2009. Clauset et al. use the Kolmogorov-...
1
vote
0answers
23 views

Connection between logit model and power law distribution

I am wondering whether it is appropriate to make a connection between the use of a logit model and a power law distribution. My dependent variable is categorical and ordinal, therefore I am fitting a ...
3
votes
1answer
129 views

Calculate Zipf-Mandelbrot parameters from distribution

I am fetching trending topics from social media where the frequency of likes is said to follow a Zipf-Mandelbrot distribution; i.e., some of the posts will have a high number of likes and some other ...
0
votes
0answers
18 views

Determining the number of observations within the tail of a given distribution

I am wondering how to determine the number of observations that fall within the tail of a distribution. I am reading a paper and the authors use the assumption that 50 observations need to fall into ...
4
votes
1answer
258 views

Characterizing/Fitting Word Count Data into Zipf / Power Law / LogNormal

Using NLTK and Pandas, I was able to process some text files and generate word count data for them, and finally create a histogram describing word frequency. However, I'm wondering what kind of ...
4
votes
2answers
190 views

Extracting power of a power law from data

My question is more about the methodology. Assuming in some experiment we have measured quantity $y$ per each unit of time $x.$ So $y$ and $x$ form our data set here. Moreover, we know that they are ...
0
votes
0answers
22 views

Does sampling an arbitrary network in a biased way make it resemble a scale-free network?

There's a recent article on Quanta (https://www.quantamagazine.org/scant-evidence-of-power-laws-found-in-real-world-networks-20180215/) as well as the primary study on ArXiv (https://arxiv.org/abs/...
2
votes
2answers
267 views

“Law of large numbers” for distribution with infinite variance?

This is a purely explorative question. I asked a question here about a "central limit theorem" for random variables with infinite variance. I did not expect it, but it turns out that even some ...
2
votes
1answer
38 views

A random variable $X$ on $(0,\infty)$ which behaves like Exp for small $x$ and Pareto for large $x$

Are there any examples of distributions which behave like Exponential for small values and like Pareto for large values. $$\ln \mathbb{P}[X>x] \sim -\lambda x, \qquad \text{ for } x \text{ small}, ...
1
vote
0answers
41 views

Mixture of power-law distribution

I have two large sets A and B of (integer) numbers, both obtained with two different (and unknown) probability distributions $\rho_A$ and $\rho_B$. A (also large) set C contains a proportion $p$ of ...
0
votes
1answer
184 views

Normalization of power-law distributed variables. Z-scores or Min-Max?

I need to make a composite index from the sum of three power-law distributed variables, which vary on different scales and have different variances. For each variable there are many observations with ...
1
vote
0answers
221 views

Generate random numbers from a power-law/exponential distribution

I'm working through the paper Power-law distributions in empirical data by Clauset, Newman and Shalizi. On page 12, they generate random numbers from the following distribution: \begin{equation*} p(...
0
votes
0answers
42 views

Proving a relationship between two variables of variable power

Suppose we have an independent variable $X$ and dependent variable $Y$. Let's assume that in reality, $Y^k$ is directly proportional to $X^n$ for $n,k>0$. If we collected a bunch of test data with ...
0
votes
0answers
274 views

Generating random samples from a power law and testing them with R igraph

I haven't looked into power laws before, so the question may very well have an obvious, embarrassing answer. In some ways it dovetails with this post. The problem may be in the method, or in the ...
0
votes
0answers
14 views

Bivariate circular power-law distribution

I am trying to model a probability distribution that maps $[0,1]^2 \rightarrow [0,1]$, i.e. each point in the unit square should be assigned a probability. The distribution should look like this: The ...
0
votes
0answers
21 views

Statistical meaning of multiply 95% interval with integral of the fit function?

I'm trying to fix some piece of code written for some statistical analysis purposes... There is one bit that I can't wrap my head around... The software is trying to do a power-law fit, say: $$ y = ...
0
votes
0answers
31 views

Log-Normal not less unequal than Power-Law

So, I read somewhere that the log-normal distribution has a more equal distribution than the power-law, but I feel that it is very hard to determine this because I cannot find an equivalence between ...
0
votes
0answers
44 views

What is a hooked powerlaw?

I happened to encounter lots of scientific/business scenarios where a Zipf/Pareto/powerlaw describes well my data. However, whenever the mean of the distribution is large enough, the fact that these ...
0
votes
0answers
64 views

Power law fitting multiple data sets simultaneously

I have several separate measurements which seem to show a trend which I would like to fit to a power law. Or perhaps any fit, the exact form is not important but the question still pertains. (I am ...
0
votes
1answer
257 views

Fitting a Pareto distribution to two dimensional data in R

I've looked here: How do I fit a set of data to a Pareto distribution in R? do-i-fit-a-set-of-data-to-a-pareto-distribution-in-r and I've checked out the poweRlaw library which is built for fitting ...
1
vote
0answers
344 views

Exponential vs Power Regression - which one is more appropriate when data points are limited?

When researching the reciprocity failure (a feature of film photography, otherwise more appropriate to https://photo.stackexchange.com/) I have run into a statistical issue. I have a datasheet for a ...
0
votes
0answers
771 views

Estimate power law exponent for node degree distribution in scale free networks

I am trying to use the powerlaw python package to estimate the power law exponent of the degree distribution in a graph. As a reference I am using ...
1
vote
0answers
301 views

Input to fit a power-law to degree distribution of a network

I would like to use R to test whether the degree distribution of a network behave like a power-law with scale-free property. Nonetheless, I've read diferent people ...
2
votes
1answer
184 views

Log-likelihood in fit_power_law{igraph}

The R package igraph has the fit_power_law function which, as you can imagine, can fit a ...
1
vote
2answers
759 views

Fitting a power law to the relationship between two variables

I have a small set of data that I need to fit a curve to (see image and data below). ...
2
votes
0answers
101 views

How to adjust your dataset so it fits a power-law? [closed]

I have created a dataset of pictures taken at the museum of different paintings. The dataset is divided into 113 different categories (paintings) and contains around 4.8k images. Just to be clear: ...
0
votes
1answer
279 views

Does my data follow power law distribution? dummy approach in R

I am trying to test if my data fits the inverse power law distribution. Specifically, I am interested if my damaged area (Shape_Area) is somehow distance dependent? I tried to apply examples from ...
0
votes
0answers
120 views

Zero-inflated + thick-tailed + unbalanced panel

I'm pretty new to statistics and need advice on how to analyse zero-inflated, thick-tailed, panel distributions. My sample is a count of enterprise births per city and per year across U.S. cities and ...
2
votes
1answer
121 views

what can I infer if my data follows power law?

I am working on data set of "Number of days taken to get a reply to mail". I have studied POWER-LAW DISTRIBUTIONS IN EMPIRICAL DATA - AARON CLAUSET. I have used powerLaw package in R to find if my ...
1
vote
0answers
19 views

Evaluating the improvement in goodness-of-fit (power function fits)

Imagine an experiment where 15 participants are presented with pictures. Each picture is presented 8 times (within-subject design). We want to evaluate the effect of picture repetition (independent ...
4
votes
1answer
264 views

Understanding power laws and log plots (Newman paper)

I'm trying to understand some of the plots in the paper Power laws, Pareto distributions and Zipf’s law by Newman. Here is the figure in question: In this paper he generates synthetic data that ...
0
votes
1answer
68 views

Linear mixed effect model for Taylor's power law: Random effects variance equal to 0

My project involves stink bug sampling on soybeans and I'm using Taylor's Power law (logvar ~ logmean) to use its parameters in the development of a sampling plan. ...
1
vote
0answers
156 views

R - power/exponential regression issue

I have the following data which I am trying to get a nice simple regression fit to it: Looks simple enough, but using R's nls function and trying both a power (ax^b+c) and an exponential (ab^x) fits ...
1
vote
1answer
192 views

Power law is Nested in Power Law with Cutoff?

I'm reading "Power-Law Distributions in Empirical Data": https://arxiv.org/pdf/0706.1062.pdf. The authors make the claim that "In some cases the distributions we wish to compare may be nested, ...
1
vote
1answer
37 views

Why is the slope not back transformed in a regression equation for allometric relationships

I'm learning about allometric relationships and how to derive the parameters from regression equations. I've seen that you can fit a linear regression model by taking the log of both the X and Y ...
3
votes
1answer
229 views

What is the discrete equivalent of the powerlaw distribution?

The question is all in the title. The only background I can add, is that I am finding a powerlaw to interpolate beautifully between my data points. Only that my data appears in integers only, so I ...
4
votes
1answer
137 views

Besides the Pareto and Zipfian distributions, which distributions obey the power-law?

I need a list of distributions that obey the power-law, beside the commonly used Pareto and Zipfian distributions. A comprehensive list or a reference to a comprehensive list will be particularly ...
7
votes
2answers
3k views

Regression for power law

This is a cross post from Math SE. I have some data (running time of an algorithm) and I think it follows a power law $$y_\mathrm{reg} = k x^a$$ I want to determine $k$ and $a$. What I have done ...
5
votes
1answer
133 views

What can we infer about the probability distribution $P(X \ge x) = {x^{- E(x)}}$

A discrete probability distribution has the following property $$ P(X \ge x) = {x^{- E(x)}} $$ where $x$ is a positive natural number and $E(X)$ is the finite expected value of the distribution. ...
6
votes
1answer
637 views

How is a negative binomial regression model different from OLS with a logged outcome variable?

I often deal with distributions of outcome variables that are highly skewed (power-law-like), and I typically want to predict these outcomes. I am wondering when it is appropriate to take a log of ...
1
vote
1answer
62 views

Parametrize fat tailed distributions

I have a empirical distribution with fat tails (i.e. $pdf(x) \sim x^{-\zeta-1}$). I want to reduce the dimensionality of this object: for any integer $k$, I want a $k-$parametric family of ...
0
votes
1answer
36 views

Can I use information about the distribution of the dependent to improve prediction?

I'm trying to make predictions about a quantity on a per-subject basis. If I aggregate my complete sample I can get very good fit for distributions like gamma or Weibull, so I can make some ...
2
votes
0answers
189 views

Power-law distribution for scale-free newtork

I am studying social network analysis for air travel, and now I am focusing my attention on the power law distribution, $P(k)=ck^{-r}$. I'm trying to estimate $r$ and $c$ with rstudio. [...
1
vote
1answer
164 views

power regression when the power is a variable

I have this function : $y = x^\alpha$ using log: $\ln(y) = \alpha\,\ln(x)$ Now, $\alpha$ itself can be decomposed and considered as a function of two variables $w_1, w_2$. We have: $\alpha = \...
1
vote
2answers
131 views

How do I show correlation with the “fat head” and “long tail” of a distribution?

I run an information booth at an airport. People come up to the booth all day and ask questions. Some questions get asked over and over again. (“Where is the rental car counter?”) Some questions are ...