Skew of log-normal distribution using sciPy

Question

I am doing bioinformatics and I am trying to fit some values to a log-normal distribution with python's sciPy version 0.11. According to the skew of the resulting distribution, I would like to make a 0-1 decision, i.e. if skew is positive, give a "0" value, as most of the data is on the left, while, if skew is negative, give a "1" value, as most of the data is on the right.

While trying to implement that, I am confused as to which formula to use for the skew. From Wikipedia, the skew is equal to: $(e^{\sigma^2}+2)\sqrt{e^{\sigma^2}-1}$, but this can never be a negative number. Using sciPy's moment method like this: scipy.stats.moment(data,3)/(std**3), seems to yield negative numbers, but then I found out this is the non-central moment, so I am afraid it is wrong.

Secondly, I have looked into this very useful question here, on how to actually fit data to the log-normal distribution. I have tried to make a toy example, where I generate 50 numbers on the interval $0.1-9$ and 1000 numbers on the interval $9-12$. So, I would expect that this distribution should have a negative skew. The code is as follows:

import numpy as np
import scipy.stats as s
from matplotlib import pyplot as plt
test1 = np.linspace(0.1,9,50)
test2 = np.linspace(9,12,1000)
test = list(test1) + list(test2)
shape,loc,scale = s.lognorm.fit(test,floc=0)
distro = s.lognorm(shape,loc,scale)
x = np.linspace(0.1,20,200)
plt.plot(x,distro.pdf(x))
plt.plot(x,distro.cdf(x))
plt.axvline(x=scale,color='r',ls='--')
plt.legend(('PDF','CDF',r'$\mu$'), loc='upper left')
plt.hist(test,bins=100,normed=True,alpha=0.4)

And the corresponding image:

My question is, does it really take that much to shift a log-normal distribution to the right? What I have done here seems to spread it wide, but shift it only a little. Should I use a different measure for my purposes?

Answer 1

According to the skew of the resulting distribution, I would like to make a 0-1 decision, i.e. if skew is positive, give a "0" value, as most of the data is on the left, while, if skew is negative, give a "1" value, as most of the data is on the right.

As a general statement, this is not true. It's often the case, but it's trivial to find counterexamples (and unfortunately many elementary books insist on making statements almost exactly like yours, to eventual substantial confusion when a real-world counterexample appears).

In your question, you're using third-moment skewness as your skewness measure, but statements that 'most of the data is on the left'/'most of the data is on the right' relates to a different kind of skewness, the second Pearson skewness coefficient.

The two measures can disagree about the sign of skewness in a population.

Consider, for example, a Poisson with mean 0.7. It has third moment skewness of about 1.195, but it has second Pearson skewness (/mean-median skewness) of $3(0.7-1)/\sqrt 0.7$, or about -1.076 (that is, in spite of having positive third moment skewness, more of its probability is above the mean than below it). [That's a simple example I only came up with a couple of days ago in response to another question; it only took a few minutes to come up with several such examples, but that one's my favourite of the counterexamples, not least because "Poisson(0.7)" is so simple to picture, so easy to remember and unambiguously convey.]

From Wikipedia, the skew is equal to: $(e^{σ^2}+2)\sqrt{e^{σ^2}−1}$, but this can never be a negative number. Using sciPy's moment method like this: scipy.stats.moment(data,3)/(std**3)

You're confusing two different things! The first is a population quantity and the second is a sample quantity.

You can have one positive while the other is negative, just by chance.

Here's an example. This is a sample I just generated from a lognormal distribution ($n=30$, with parameters $\mu=0$ and $\sigma=0.1$), which happens to have negative sample skewness:

 0.139  0.086  0.046 -0.084  0.020  0.050 -0.041  0.080 -0.048  0.076 -0.050  0.023  
 0.063 -0.210  0.011 -0.343 -0.016 -0.005  0.123  0.044 -0.026  0.048  0.107  0.066  
 0.089 -0.047  0.175 -0.092 -0.095  0.020

In fact with those parameters and that sample size, the sample third moment is negative almost half the time.