Comparison of medians

I`m currently an Intern and one of my tasks is to asses whether certain variables were optimal for a control system. In total there are four variables which can vary, and to avoid a complicated analysis I decided to, from the default settings (as a control), only vary one of these variables at a time and see how it impacted the stabilization time of the system. Each of these sample consists of 2500 tests and the time it takes to converge for each test. Just to clarify, my values are discrete as time is the number of cycles it takes to converge.

This approach seemed simple enough, however after preliminary analysis it became apparent that the distributions were not normal and that one of the parameters played heavily with the skew and shape of the PDF. Non-parametric tests seemed most-appropriate and I felt the median would be most representative of the time it took to settle, due to the heavy skew. The Mann-Whitney test seemed perfect, but the assumptions it required to be valid to compare the medians were too restrictive (My data is discrete and the distributions do not appear to just be shifted). The next test I thought of using was a permutation test to compare the medians of our two distributions, however I have no Idea how to conclude from the distribution of the medians. I though about just noting the percentage of the time that one settled faster than the other, but have no idea how to prove if that is statistically significant.

Edit: Ok, minutes after having posted something clicked and now everything seems clear. My question now is rather does it make sense or is there a better approach?

Histograms

The first histogram is that of my control settings (epsilon = 0.1) and the other is when epsilon = 0.4. As epsilon grows the data's skew tends to become (from what I can see) more and more to the right. Due to my lack of reputation points I can only post two images, but the trends persists for all the values of epsilon I have tested thus far.

• What do you intend by "non-parametric distribution" here? Aug 4, 2016 at 0:37
• I meant that I didn't manage to fit it to a known distribution. I'll include some pictures, I think it will answer your question better than I can. Aug 4, 2016 at 13:57
• Even if you did fit a known distribution to your sample, it didn't actually come from that distribution; if the word "nonparametric" is simply to indicate we don't know what distribution(s) the data are from, it seems that it's not performing a useful role. We can almost never know that. Aug 4, 2016 at 16:21
• Ok, how would you say that instead? My exposure to statistics has been rather limited thus far, so any help would be greatly appreciated! Aug 4, 2016 at 17:53
• Comparison of medians (if you really want a comparison of medians) or some similar terminology if you actually want to compare something else. Aug 4, 2016 at 22:36