2
$\begingroup$

EDIT:

Perhaps I should also note that my initial attempt at analyzing this data used a hierarchical model in rJAGS sampling the means from uniform distributions and variances from gamma distributions but I found the choice of priors on the group level variance heavily influenced the results. Perhaps there is a better way to use that approach to estimate? Any advice on the best way to do this would be welcome.

End Edit

This question is related to these two previous questions of mine. @Elvis has suggested I turn it into its own question as the comments were getting lengthy:

How to use profile likelihood?

What is the relationship between profile likelihood and confidence intervals?

I have data like the following based on animals performing a behavioral task:

1) Data can take any value from 0 to 20

2) Data from a relatively large number (~50) of previous results with control animals that indicates the expected distribution of results (at least for the control group) is not normal and looks like the top plot.

3) I have new data shown in the boxplots in the middle panel. Group 1 is a control that should be the same as the previous results (although no two groups of animals and conditions are ever exactly the same). Group 2 is a control that received placebo treatment which is expected to have no effect, thus it should be the same as Group 1. Group 3 received a drug.

4) The goal of the study is to determine if the drug may be affecting the behavior of the animals.

5) There are many other factors that may affect the behavior of animals so it is the common assumption that the best that can be done is see if the drug affected the behavior on average. Thus I wish to compare the average outcomes of each group.

6) Previous studies have compared using one-way ANOVA and post-hoc t-tests then reporting p-values. This bothers me as the assumptions of the tests (normality) appear to be violated, using an arbitrary cutoff to decide "significance" (eg p<0.05) seems illogical, and I would never expect any two groups of animals to be exactly the same (null hypothesis of two means are equal is always false). Therefore I would like to instead simply estimate the means and show what means are most likely for each group using likelihoods and say something like the mean of group 3 is 20x more likely to be 15 than the mean of group 1.

7) I may also/instead want to calculate confidence intervals for the means. It is not clear to me whether this is something I should want to do.

8) Note that this is not my actual data but was instead gotten by sampling from the "prior/previous" density, eg: sample(x=prior.dens$x,size=12, prob=prior.dens$y) So the appearance of a true difference between groups is false here. However this is very similar to the situation with my real data.

9) I have calculated the profile likelihood (lower plot) using this R code which appears to be valid based on the responses to my previous questions:

muVals <- seq(0,20, length = 10000)
profile.likelihood<-function(dat, muVals){
  likVals <- sapply(muVals,
                    function(mu){
                      (sum((dat - mu)^2) /
                         sum((dat - mean(dat))^2)) ^ (-length(dat)/2)
                    }
  )

return(cbind(muVals,likVals))
}

enter image description here

dput() of new data:

structure(c(1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 
2, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 13.6986301369863, 
16.1643835616438, 12.0547945205479, 12.1722113502935, 9.74559686888454, 
0.430528375733855, 11.3502935420744, 10.6457925636008, 9.9412915851272, 
10.7240704500978, 10.958904109589, 11.6242661448141, 17.9701232444495, 
15.9326690901071, 7.98247314058244, 14.4031004607677, 13.5198221541941, 
2.82421704847366, 16.114045586437, 19.328767512925, 3.74181577004492, 
17.4085859861225, 19.2017483590171, 8.26946665905416, 10.0207103956491, 
16.1689247898757, 13.9989542039111, 9.80047978740185, 17.8596440777183, 
18.1706223106012, 18.1891529858112, 11.7567204562947), .Dim = c(32L, 
2L))

dput() of prior density

structure(list(x = c(0, 0.0391389432485323, 0.0782778864970646, 
0.117416829745597, 0.156555772994129, 0.195694716242661, 0.234833659491194, 
0.273972602739726, 0.313111545988258, 0.352250489236791, 0.391389432485323, 
0.430528375733855, 0.469667318982387, 0.50880626223092, 0.547945205479452, 
0.587084148727984, 0.626223091976517, 0.665362035225049, 0.704500978473581, 
0.743639921722113, 0.782778864970646, 0.821917808219178, 0.86105675146771, 
0.900195694716243, 0.939334637964775, 0.978473581213307, 1.01761252446184, 
1.05675146771037, 1.0958904109589, 1.13502935420744, 1.17416829745597, 
1.2133072407045, 1.25244618395303, 1.29158512720157, 1.3307240704501, 
1.36986301369863, 1.40900195694716, 1.44814090019569, 1.48727984344423, 
1.52641878669276, 1.56555772994129, 1.60469667318982, 1.64383561643836, 
1.68297455968689, 1.72211350293542, 1.76125244618395, 1.80039138943249, 
1.83953033268102, 1.87866927592955, 1.91780821917808, 1.95694716242661, 
1.99608610567515, 2.03522504892368, 2.07436399217221, 2.11350293542074, 
2.15264187866928, 2.19178082191781, 2.23091976516634, 2.27005870841487, 
2.30919765166341, 2.34833659491194, 2.38747553816047, 2.426614481409, 
2.46575342465753, 2.50489236790607, 2.5440313111546, 2.58317025440313, 
2.62230919765166, 2.6614481409002, 2.70058708414873, 2.73972602739726, 
2.77886497064579, 2.81800391389432, 2.85714285714286, 2.89628180039139, 
2.93542074363992, 2.97455968688845, 3.01369863013699, 3.05283757338552, 
3.09197651663405, 3.13111545988258, 3.17025440313112, 3.20939334637965, 
3.24853228962818, 3.28767123287671, 3.32681017612524, 3.36594911937378, 
3.40508806262231, 3.44422700587084, 3.48336594911937, 3.52250489236791, 
3.56164383561644, 3.60078277886497, 3.6399217221135, 3.67906066536204, 
3.71819960861057, 3.7573385518591, 3.79647749510763, 3.83561643835616, 
3.8747553816047, 3.91389432485323, 3.95303326810176, 3.99217221135029, 
4.03131115459883, 4.07045009784736, 4.10958904109589, 4.14872798434442, 
4.18786692759295, 4.22700587084149, 4.26614481409002, 4.30528375733855, 
4.34442270058708, 4.38356164383562, 4.42270058708415, 4.46183953033268, 
4.50097847358121, 4.54011741682975, 4.57925636007828, 4.61839530332681, 
4.65753424657534, 4.69667318982387, 4.73581213307241, 4.77495107632094, 
4.81409001956947, 4.853228962818, 4.89236790606654, 4.93150684931507, 
4.9706457925636, 5.00978473581213, 5.04892367906066, 5.0880626223092, 
5.12720156555773, 5.16634050880626, 5.20547945205479, 5.24461839530333, 
5.28375733855186, 5.32289628180039, 5.36203522504892, 5.40117416829746, 
5.44031311154599, 5.47945205479452, 5.51859099804305, 5.55772994129159, 
5.59686888454012, 5.63600782778865, 5.67514677103718, 5.71428571428571, 
5.75342465753425, 5.79256360078278, 5.83170254403131, 5.87084148727984, 
5.90998043052838, 5.94911937377691, 5.98825831702544, 6.02739726027397, 
6.0665362035225, 6.10567514677104, 6.14481409001957, 6.1839530332681, 
6.22309197651663, 6.26223091976517, 6.3013698630137, 6.34050880626223, 
6.37964774951076, 6.4187866927593, 6.45792563600783, 6.49706457925636, 
6.53620352250489, 6.57534246575342, 6.61448140900196, 6.65362035225049, 
6.69275929549902, 6.73189823874755, 6.77103718199609, 6.81017612524462, 
6.84931506849315, 6.88845401174168, 6.92759295499021, 6.96673189823875, 
7.00587084148728, 7.04500978473581, 7.08414872798434, 7.12328767123288, 
7.16242661448141, 7.20156555772994, 7.24070450097847, 7.27984344422701, 
7.31898238747554, 7.35812133072407, 7.3972602739726, 7.43639921722114, 
7.47553816046967, 7.5146771037182, 7.55381604696673, 7.59295499021526, 
7.6320939334638, 7.67123287671233, 7.71037181996086, 7.74951076320939, 
7.78864970645793, 7.82778864970646, 7.86692759295499, 7.90606653620352, 
7.94520547945205, 7.98434442270059, 8.02348336594912, 8.06262230919765, 
8.10176125244618, 8.14090019569472, 8.18003913894325, 8.21917808219178, 
8.25831702544031, 8.29745596868884, 8.33659491193738, 8.37573385518591, 
8.41487279843444, 8.45401174168297, 8.49315068493151, 8.53228962818004, 
8.57142857142857, 8.6105675146771, 8.64970645792564, 8.68884540117417, 
8.7279843444227, 8.76712328767123, 8.80626223091977, 8.8454011741683, 
8.88454011741683, 8.92367906066536, 8.96281800391389, 9.00195694716243, 
9.04109589041096, 9.08023483365949, 9.11937377690802, 9.15851272015655, 
9.19765166340509, 9.23679060665362, 9.27592954990215, 9.31506849315068, 
9.35420743639922, 9.39334637964775, 9.43248532289628, 9.47162426614481, 
9.51076320939335, 9.54990215264188, 9.58904109589041, 9.62818003913894, 
9.66731898238747, 9.70645792563601, 9.74559686888454, 9.78473581213307, 
9.8238747553816, 9.86301369863014, 9.90215264187867, 9.9412915851272, 
9.98043052837573, 10.0195694716243, 10.0587084148728, 10.0978473581213, 
10.1369863013699, 10.1761252446184, 10.2152641878669, 10.2544031311155, 
10.293542074364, 10.3326810176125, 10.3718199608611, 10.4109589041096, 
10.4500978473581, 10.4892367906067, 10.5283757338552, 10.5675146771037, 
10.6066536203522, 10.6457925636008, 10.6849315068493, 10.7240704500978, 
10.7632093933464, 10.8023483365949, 10.8414872798434, 10.880626223092, 
10.9197651663405, 10.958904109589, 10.9980430528376, 11.0371819960861, 
11.0763209393346, 11.1154598825832, 11.1545988258317, 11.1937377690802, 
11.2328767123288, 11.2720156555773, 11.3111545988258, 11.3502935420744, 
11.3894324853229, 11.4285714285714, 11.46771037182, 11.5068493150685, 
11.545988258317, 11.5851272015656, 11.6242661448141, 11.6634050880626, 
11.7025440313112, 11.7416829745597, 11.7808219178082, 11.8199608610568, 
11.8590998043053, 11.8982387475538, 11.9373776908023, 11.9765166340509, 
12.0156555772994, 12.0547945205479, 12.0939334637965, 12.133072407045, 
12.1722113502935, 12.2113502935421, 12.2504892367906, 12.2896281800391, 
12.3287671232877, 12.3679060665362, 12.4070450097847, 12.4461839530333, 
12.4853228962818, 12.5244618395303, 12.5636007827789, 12.6027397260274, 
12.6418786692759, 12.6810176125245, 12.720156555773, 12.7592954990215, 
12.7984344422701, 12.8375733855186, 12.8767123287671, 12.9158512720157, 
12.9549902152642, 12.9941291585127, 13.0332681017613, 13.0724070450098, 
13.1115459882583, 13.1506849315068, 13.1898238747554, 13.2289628180039, 
13.2681017612524, 13.307240704501, 13.3463796477495, 13.385518590998, 
13.4246575342466, 13.4637964774951, 13.5029354207436, 13.5420743639922, 
13.5812133072407, 13.6203522504892, 13.6594911937378, 13.6986301369863, 
13.7377690802348, 13.7769080234834, 13.8160469667319, 13.8551859099804, 
13.894324853229, 13.9334637964775, 13.972602739726, 14.0117416829746, 
14.0508806262231, 14.0900195694716, 14.1291585127202, 14.1682974559687, 
14.2074363992172, 14.2465753424658, 14.2857142857143, 14.3248532289628, 
14.3639921722113, 14.4031311154599, 14.4422700587084, 14.4814090019569, 
14.5205479452055, 14.559686888454, 14.5988258317025, 14.6379647749511, 
14.6771037181996, 14.7162426614481, 14.7553816046967, 14.7945205479452, 
14.8336594911937, 14.8727984344423, 14.9119373776908, 14.9510763209393, 
14.9902152641879, 15.0293542074364, 15.0684931506849, 15.1076320939335, 
15.146771037182, 15.1859099804305, 15.2250489236791, 15.2641878669276, 
15.3033268101761, 15.3424657534247, 15.3816046966732, 15.4207436399217, 
15.4598825831703, 15.4990215264188, 15.5381604696673, 15.5772994129159, 
15.6164383561644, 15.6555772994129, 15.6947162426614, 15.73385518591, 
15.7729941291585, 15.812133072407, 15.8512720156556, 15.8904109589041, 
15.9295499021526, 15.9686888454012, 16.0078277886497, 16.0469667318982, 
16.0861056751468, 16.1252446183953, 16.1643835616438, 16.2035225048924, 
16.2426614481409, 16.2818003913894, 16.320939334638, 16.3600782778865, 
16.399217221135, 16.4383561643836, 16.4774951076321, 16.5166340508806, 
16.5557729941292, 16.5949119373777, 16.6340508806262, 16.6731898238748, 
16.7123287671233, 16.7514677103718, 16.7906066536204, 16.8297455968689, 
16.8688845401174, 16.9080234833659, 16.9471624266145, 16.986301369863, 
17.0254403131115, 17.0645792563601, 17.1037181996086, 17.1428571428571, 
17.1819960861057, 17.2211350293542, 17.2602739726027, 17.2994129158513, 
17.3385518590998, 17.3776908023483, 17.4168297455969, 17.4559686888454, 
17.4951076320939, 17.5342465753425, 17.573385518591, 17.6125244618395, 
17.6516634050881, 17.6908023483366, 17.7299412915851, 17.7690802348337, 
17.8082191780822, 17.8473581213307, 17.8864970645793, 17.9256360078278, 
17.9647749510763, 18.0039138943249, 18.0430528375734, 18.0821917808219, 
18.1213307240705, 18.160469667319, 18.1996086105675, 18.238747553816, 
18.2778864970646, 18.3170254403131, 18.3561643835616, 18.3953033268102, 
18.4344422700587, 18.4735812133072, 18.5127201565558, 18.5518590998043, 
18.5909980430528, 18.6301369863014, 18.6692759295499, 18.7084148727984, 
18.747553816047, 18.7866927592955, 18.825831702544, 18.8649706457926, 
18.9041095890411, 18.9432485322896, 18.9823874755382, 19.0215264187867, 
19.0606653620352, 19.0998043052838, 19.1389432485323, 19.1780821917808, 
19.2172211350294, 19.2563600782779, 19.2954990215264, 19.3346379647749, 
19.3737769080235, 19.412915851272, 19.4520547945205, 19.4911937377691, 
19.5303326810176, 19.5694716242661, 19.6086105675147, 19.6477495107632, 
19.6868884540117, 19.7260273972603, 19.7651663405088, 19.8043052837573, 
19.8434442270059, 19.8825831702544, 19.9217221135029, 19.9608610567515, 
20), y = c(0.0217667400980039, 0.0227118960613568, 0.0236399642746827, 
0.024547462294379, 0.0254284817500052, 0.0262697990497476, 0.0270761925341352, 
0.0278447201687443, 0.0285725349842736, 0.0292431898522125, 0.0298599237216537, 
0.0304262344415342, 0.0309405476728982, 0.0313967502030459, 0.0317805594200756, 
0.0321087524689367, 0.0323813637754389, 0.0325985887432935, 0.0327456085503099, 
0.0328338748111955, 0.032871094252979, 0.032858917719392, 0.0327952170079012, 
0.0326741710852976, 0.0325130172841641, 0.0323142824586605, 0.0320805363813515, 
0.0318066450090944, 0.0315032349150802, 0.0311762923391493, 0.0308283666713486, 
0.0304606397730375, 0.0300742685852501, 0.0296767520098838, 0.0292699115474762, 
0.0288554692880644, 0.0284340054317109, 0.0280086390955194, 0.0275806684358679, 
0.0271507171781247, 0.0267191796498216, 0.0262860885684517, 0.0258516152315375, 
0.025415436557538, 0.0249771598811644, 0.0245350861062508, 0.0240885560431356, 
0.023637102870035, 0.023179973519991, 0.0227156094045636, 0.022241184162465, 
0.0217581061784532, 0.0212659006410898, 0.0207641513896269, 0.020249460771043, 
0.0197238756753159, 0.0191886405031699, 0.0186441419420248, 0.0180900824959544, 
0.0175265997231419, 0.0169574756929679, 0.0163840336778012, 0.0158076842167135, 
0.0152307532198712, 0.0146567117250279, 0.0140876768708212, 0.0135257717396179, 
0.0129746875992836, 0.0124411735382691, 0.0119248408581489, 0.0114281180660412, 
0.0109534391260541, 0.0105125675916208, 0.0101040127121235, 0.0097278039838056, 
0.00938617839440894, 0.00908600262449913, 0.00883795563273938, 
0.00863315495962393, 0.00847325872834633, 0.00835985054125376, 
0.0083104707966061, 0.00831574579983596, 0.00837242006745159, 
0.0084811812661746, 0.00864895114585229, 0.0088847277032945, 
0.0091733846139949, 0.00951451815849311, 0.00990760864554539, 
0.0103676944983387, 0.0108803185070754, 0.0114398691731338, 0.0120446169019066, 
0.0126975354786556, 0.0134003348410125, 0.014138322350119, 0.0149085888122459, 
0.0157081161739461, 0.0165397010226779, 0.0173908415086893, 0.0182558660392905, 
0.0191308446396648, 0.0200115146141704, 0.0208896684310384, 0.0217595621107009, 
0.0226168398271489, 0.0234571367239625, 0.0242658576551506, 0.0250415384796808, 
0.025781743996249, 0.0264824465880188, 0.0271330271813973, 0.0277197602228349, 
0.0282514207578494, 0.0287250463560853, 0.0291378145433322, 0.0294643336036007, 
0.0297185497373367, 0.0299034233614394, 0.0300178272409036, 0.0300510165057044, 
0.0299933390096322, 0.029864296171915, 0.0296646737354628, 0.0293954446422498, 
0.02903609387537, 0.0286087954739586, 0.0281202389214091, 0.0275730852809592, 
0.0269631864592354, 0.0262908755920268, 0.025574205859082, 0.0248169722677519, 
0.024023046736675, 0.0231883685262104, 0.0223289370148913, 0.0214507109581384, 
0.0205578894465608, 0.0196543161993343, 0.0187475783069642, 0.0178440120329414, 
0.0169474708137715, 0.0160617456778675, 0.0151982413151739, 0.0143579764501946, 
0.0135431035900457, 0.0127565541603801, 0.0120065112945572, 0.0113013352426194, 
0.010635019980095, 0.0100094469431421, 0.00942639127892244, 0.0089039890191574, 
0.00843070966487366, 0.00800519340240519, 0.00762818830262024, 
0.00730797543721608, 0.00704980283664362, 0.00684141502281347, 
0.00668265968608497, 0.00657330236985191, 0.00652994734551145, 
0.00653626373538692, 0.00658916348197513, 0.00668767640137901, 
0.00683740677506747, 0.00703949814277543, 0.00728169259995597, 
0.00756244741730784, 0.00788017305139687, 0.00824447686134098, 
0.00864128336177512, 0.00906733428314206, 0.00952075397149063, 
0.0100031836945359, 0.010512665615019, 0.0110414546541446, 0.0115878311927959, 
0.0121501195536992, 0.0127308183673382, 0.01332332344487, 0.0139262209784856, 
0.0145387177488175, 0.015161575268352, 0.0157951749967284, 0.016438040193859, 
0.0170909905985346, 0.0177550383014629, 0.0184378178562425, 0.019138615981402, 
0.0198599761627206, 0.0206051085684539, 0.0213835326226107, 0.0222057991861808, 
0.0230705544613072, 0.0239831651499172, 0.0249491866774483, 0.0260019858815712, 
0.0271307566962957, 0.0283404432091247, 0.0296377725554787, 0.0310483777483443, 
0.0325909750094088, 0.0342501536507845, 0.0360320721161026, 0.0379427304435736, 
0.0400427505040893, 0.0422911469270196, 0.0446880555428848, 0.0472364098981542, 
0.0499665903504324, 0.0528865496293085, 0.0559605500295509, 0.0591864336230866, 
0.0625614652392627, 0.0661323901936574, 0.0698369942423732, 0.0736642649820459, 
0.0776053170213216, 0.0816654985557755, 0.0858243890440125, 0.0900462454820139, 
0.0943168565446659, 0.0986215759610166, 0.102937806780721, 0.107234576343045, 
0.111494854705369, 0.115701725627134, 0.119818728151249, 0.123810263483841, 
0.127676838260079, 0.131403340542788, 0.134975040188428, 0.138301149786118, 
0.141429558148372, 0.144353043040705, 0.147063324586097, 0.149509011012514, 
0.151678338013187, 0.153617609445566, 0.155327745461799, 0.156810642833852, 
0.157987822962458, 0.15895731472489, 0.159731001895812, 0.160320192816835, 
0.16070927729284, 0.160927335641176, 0.161026262422779, 0.161024035352265, 
0.160939101818898, 0.160783676919756, 0.160610951636628, 0.160440813332408, 
0.160292840283904, 0.160202196337883, 0.160205045445256, 0.160307691092289, 
0.160525158759831, 0.160872099587403, 0.161427322298974, 0.162146341126922, 
0.16303494787568, 0.1640978213177, 0.165374536988198, 0.166861439709314, 
0.168517967752217, 0.170337253164731, 0.172312150290867, 0.174476220294316, 
0.17675201562322, 0.17912237546253, 0.181569351447975, 0.184077063209181, 
0.186601937519015, 0.189110996532675, 0.19158078566652, 0.193986444864657, 
0.196248038237016, 0.198366671014338, 0.2003193238609, 0.202083414374645, 
0.203583109911179, 0.204784727823111, 0.20571235693773, 0.206350205706743, 
0.206678507619118, 0.206556713002678, 0.206095969055307, 0.205290936474901, 
0.20413741111321, 0.202556572638439, 0.20056355298721, 0.198228613635869, 
0.195558644146613, 0.192555647948946, 0.189126333152985, 0.185410492636506, 
0.181423717899618, 0.177182210388674, 0.172658716028772, 0.167900183255391, 
0.16296802586402, 0.157882368007903, 0.152661212543965, 0.147304635977954, 
0.141882524743064, 0.136414358489409, 0.130919304274034, 0.125422474679471, 
0.119957697079395, 0.11454052598248, 0.109185666974252, 0.103909993572189, 
0.0987683404859644, 0.093741308688079, 0.0888376270971578, 0.0840654236882268, 
0.0794666168513133, 0.0750427325158831, 0.0707727313029462, 0.0666591512693038, 
0.0627091344513033, 0.0589774733641864, 0.0554044324211085, 0.0519885093076509, 
0.0487279204345587, 0.0456550213639223, 0.0427535393057037, 0.0399945562439461, 
0.0373743649778989, 0.0348937863392637, 0.0325878723180957, 0.0304033659564744, 
0.0283361981437062, 0.0263823330082348, 0.0245619867105529, 0.0228595658836749, 
0.0212541351296348, 0.0197423154392725, 0.0183242669772017, 0.0170220160814398, 
0.0158005734119126, 0.0146575124941681, 0.0135905002875402, 0.0126143097870055, 
0.0117191044664318, 0.0108918081116421, 0.0101309359870987, 0.0094379000631823, 
0.00883019944826195, 0.00828380391399826, 0.00779783511360308, 
0.00737145971429946, 0.00701812393006059, 0.0067311252542151, 
0.00650097052493056, 0.00632717374230014, 0.00621198669319391, 
0.00617143557767508, 0.0061855594050197, 0.00625405591658972, 
0.00637663424018689, 0.00656643085439909, 0.00681747969859399, 
0.00712155184640286, 0.00747843433306984, 0.0078907086989986, 
0.00837340291572697, 0.00890806427672648, 0.009494513737272, 
0.010132574128414, 0.010835143782014, 0.0115961949265227, 0.0124081127363209, 
0.0132707231223347, 0.0141867435373067, 0.0151701717571193, 0.0162034512423126, 
0.0172863373562776, 0.0184185690183941, 0.019612438019485, 0.0208613703536178, 
0.0221579723104729, 0.0235016744919049, 0.0248946535533411, 0.0263479485897112, 
0.0278446638511839, 0.0293835133691365, 0.0309631084740302, 0.0325912348690895, 
0.0342591472113346, 0.0359590659180276, 0.0376882779600787, 0.0394452615206507, 
0.0412293518464991, 0.043027543361404, 0.0448353514472685, 0.0466481043572906, 
0.0484577337141087, 0.0502538712990142, 0.0520305447610213, 0.0537813149370389, 
0.0554964047538021, 0.0571515262745735, 0.0587502027060215, 0.0602849490847391, 
0.0617482500354599, 0.0631066082295335, 0.0643579690671984, 0.0655057363703737, 
0.0665429490020974, 0.0674531640036809, 0.0681929810868339, 0.0687958988058021, 
0.0692572514303207, 0.0695726946039624, 0.069695167044965, 0.0696433847901905, 
0.0694350823030356, 0.0690697581056959, 0.068535116834938, 0.0677961078509799, 
0.0669066143489608, 0.0658704054221769, 0.0646916508125923, 0.0633398550663818, 
0.0618485040466386, 0.0602410483121883, 0.0585250659382392, 0.0567013372657685, 
0.0547654519871085, 0.0527582945347469, 0.0506891518402187, 0.0485673654999008, 
0.0463957642449994, 0.0441983057887947, 0.0419884488508511, 0.0397750260303709, 
0.0375684836047041, 0.0353879440268247, 0.0332384672584751, 0.0311267921300081, 
0.0290593901126105, 0.0270601872571371, 0.0251287992452735, 0.0232632081303122, 
0.0214669543868477, 0.0197503127220639, 0.0181339971366657, 0.0165964380609598, 
0.0151383742283497, 0.013760299683845, 0.012485591080821, 0.0112979221667829, 
0.0101874296294859, 0.00915261833812058, 0.00819857079937494, 
0.00733611052320258, 0.00654025398573884, 0.00580844808959148, 
0.0051380838601794, 0.00454221827421074, 0.00400446040436533, 
0.00351615254678973, 0.0030745243393893, 0.00268072850148871, 
0.00233768602660161, 0.00202984861125548, 0.00175479227131742, 
0.00151014383233181, 0.00130072566668639, 0.00111721604283242, 
0.000955112980316734, 0.000812623159815952, 0.000689590092917738, 
0.0005860912862362, 0.000495639552545513, 0.000417014922709681, 
0.000349049284657548, 0.000292958836410902, 0.00024515801710463, 
0.000204040052154968, 0.000168884867932852, 0.000139474416390484, 
0.000115548025328669, 9.51646202245106e-05, 7.79159203801693e-05, 
6.34177645960676e-05, 5.18748729574288e-05, 4.22967579760247e-05, 
3.42684953287067e-05, 2.75891372479148e-05, 2.21737075433437e-05, 
1.79085413159829e-05, 1.43641470627961e-05, 1.14430817192601e-05, 
9.05508126397165e-06, 7.22026287729582e-06, 5.73639534240461e-06, 
4.52392445545382e-06, 3.54207412160509e-06, 2.77052765004019e-06, 
2.18172878629954e-06, 1.7042057736597e-06, 1.32082991902191e-06, 
1.01596818604684e-06, 7.89867561242482e-07, 6.11506981747829e-07, 
4.69391759719292e-07, 3.57363517027134e-07), bw = 0.715374250961732, 
    n = 35L, call = density.default(x = dat.prior, from = 0, 
        to = 20), data.name = "dat.prior", has.na = FALSE), .Names = c("x", 
"y", "bw", "n", "call", "data.name", "has.na"), class = "density")
$\endgroup$

1 Answer 1

2
$\begingroup$

First of all, visual inspection of your barplot shows that there might be a difference between group 1 (control group) and the two other groups, but that the drug and placebo groups seem identical... The next remark is that your samples are very small, and the dispersion seems high, which will make difficult to detect a potential effect.

Now a qq-plot of your group 1 (control group) against your prior distribution. I did a KS-test also: it rejects the hypothesis that the control group is drawn from the prior distribution, but I think that the qq-plot is sort of comforting.

enter image description here

Now two group comparisons, a parametric (A contains your "new data")

> summary( lm( A[,2] ~ as.factor(A[,1]) ))

Call:
lm(formula = A[, 2] ~ as.factor(A[, 1]))

Residuals:  
   Min       1Q   Median       3Q      Max 
-10.3620  -1.4320   0.7145   3.2726   6.4062 

Coefficients:    
               Estimate Std. Error t value Pr(>|t|)    
(Intercept)          10.793      1.337   8.073 6.66e-09 ***
as.factor(A[, 1])2    2.130      1.983   1.074   0.2916    
as.factor(A[, 1])3    3.551      1.983   1.791   0.0838 .  
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 

Residual standard error: 4.631 on 29 degrees of freedom
Multiple R-squared: 0.102,  Adjusted R-squared: 0.04003 
F-statistic: 1.646 on 2 and 29 DF,  p-value: 0.2103 

And a non-parametric (does not use normality assumption):

> kruskal.test( A[,2], as.factor(A[,1]) )

    Kruskal-Wallis rank sum test

data:  A[, 2] and as.factor(A[, 1]) 
Kruskal-Wallis chi-squared = 3.5521, df = 2, p-value = 0.1693

I don’t think there’s any hope to find a better result using other tools. Hopefully someone will comment and find a big mistake in my analyses!

$\endgroup$
5
  • $\begingroup$ Thanks, but part of my problem is that I think the presence of a difference itself cannot be attributed to treatment so is not of interest. I wanted to see if using likelihoods was a better way of estimating the group means. $\endgroup$
    – Flask
    Commented Dec 12, 2013 at 13:54
  • $\begingroup$ The high dispersion of your data makes the use of the empirical mean hazardous. The median is more robust and I think you should use it here. $\endgroup$
    – Elvis
    Commented Dec 12, 2013 at 22:29
  • $\begingroup$ Actually that is another worry I have had. The mean is supported by the central limit theorem. $\endgroup$
    – Flask
    Commented Dec 18, 2013 at 1:29
  • $\begingroup$ In fact as your distribution is bounded, its variance is finite so the central limit theorem holds... If you believe firmly in your "prior distribution", one can estimate $\sigma^2 = 15.3$, and for $n = 10$ you could apply CLT... the confidence interval would then be like mean$\pm 1.96 \times \sqrt{15.3/10}$ that is mean$\pm 2.42$ which is huge. BUT your samples 2 and 3 really don’t seem to be drawn from the prior distribution (you may want to test this!) and using quantiles of $t(9)$ with the empirical variance you will get something like mean$\pm 3$... $\endgroup$
    – Elvis
    Commented Dec 18, 2013 at 1:52
  • $\begingroup$ See this for a simple way to get CI on the median. In fact I am not sure it will be smaller than the CI for the mean... $\endgroup$
    – Elvis
    Commented Dec 18, 2013 at 1:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.