1
$\begingroup$

The data

The dataset comprises 10 variables: waiting times (rounded to minutes) for questions to be answered on Stack Overflow by programming language. Each is discrete, none is normally distributed and all are different in size. The conditional means and variances indicate significant over-dispersion.

The aim

Following some survival analysis around probabilities I want to determine if the differences in waiting times between the languages are significant. Having researched the non-applicability of $t$-tests and Wilcoxon-Mann-Whitney tests for discrete data, and explored some of the sensitivities around sample sizes and degrees of skew, I'm at a loss trying to find the most appropriate method.

I experimented with a Negative Binomial regression using the language with the quickest answer time, c++, as the dependent variable. Results below. The idea came from this paper which advises using Poisson or Negative Binomial regression with discrete and/or highly skewed data. However, as I have 9 independent variables I'm unsure of the appropriateness of these results as measures of significance.

Positively, the coefficients match the intuition I developed through EDA. 

                         Generalized Linear Model Regression Results                  
==============================================================================
Dep. Variable:             c_plusplus   No. Observations:                25021
Model:                            GLM   Df Residuals:                    25011
Model Family:        NegativeBinomial   Df Model:                            9
Link Function:                    log   Scale:                  0.129917576134
Method:                          IRLS   Log-Likelihood:                    nan
Date:                Mon, 12 Oct 2015   Deviance:                          nan
Time:                        20:13:29   Pearson chi2:                 3.25e+03
No. Iterations:                    20                                         
==============================================================================
                 coef    std err          z      P>|z|      [95.0% Conf. Int.]
------------------------------------------------------------------------------
Intercept      2.0341      0.008    265.033      0.000         2.019     2.049
aspunet      -13.8595      6.856     -2.022      0.043       -27.296    -0.423
c_sharp      -14.7241      5.542     -2.657      0.008       -25.586    -3.863
iphone       -12.0858      7.266     -1.663      0.096       -26.327     2.155
java         -14.0126      6.347     -2.208      0.027       -26.452    -1.573
javascript   -14.4314      9.903     -1.457      0.145       -33.842     4.979
jquery       -13.8768      9.744     -1.424      0.154       -32.975     5.221
php          -14.3465      8.592     -1.670      0.095       -31.187     2.494
python       -13.7037      7.712     -1.777      0.076       -28.818     1.411
unet         -14.4260      9.268     -1.556      0.120       -32.591     3.739
==============================================================================

The questions:

  1. Can I use the coefficients and p-values from this regression model as measures of significance between the language answer times?

  2. If not, what would be a better approach?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

Waiting time should not be discrete. It's time, how can it be discrete? Even in quantum physics time is continuous. My first choice in modeling the waiting time would be exponential distribution. It's the same underlying process as Poisson. The latter gets number of occurrences in a given time period (discrete), the former is the time between occurrences (continuous) at given rate $\lambda$ of occurrences per unit of time.

What to do? I'd suggest instead of fixating on variances, to go after the distributions. Look at Kruskall-Wallis test, for instance. You could plug your data sets into the test, and it'll test whether they come from the same distribution. Kolmogorov-Smirnov test can do a similar analysis.

Also, look at Mood's Median Test, which has different assumptions from KW test, particularly, it works better when variances are very different between samples.

Improve this answer
$\endgroup$
4
  • $\begingroup$ You're right, and I'm sorry, I should have explained the following in the question. The data are are rounded to minutes. The data can be counted and listed; thereby I think I'm right in saying the data are discrete. I've updated the question to reflect this. $\endgroup$
    – RDJ
    Oct 12, 2015 at 20:15
  • 1
    $\begingroup$ @Jonathan, you're right that data is discrete, but it doesn't make it suitable for discrete distribution modeling. Your underlying process is not discrete, it's just the data precision makes it look discrete. However, this can be said about anything represented in computers or on paper. If you round to N digits, anything will become discrete, yet the underlying process is still continuous. $\endgroup$
    – Aksakal
    Oct 12, 2015 at 20:17
  • $\begingroup$ I've read around the Kruskall-Wallis and Mood's Median tests. Pleasingly, both work perfectly. Thanks. You have taught me a lot today. $\endgroup$
    – RDJ
    Oct 12, 2015 at 21:22
  • 1
    $\begingroup$ @Jonathan, I hope you'll share the results of your study somewhere. I'm curious to know if C++ questions are longer to answer than Java. $\endgroup$
    – Aksakal
    Oct 12, 2015 at 21:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.