# Why am I seeing a low F-statistic, but a very significant p-value in permanova?

So I am having some numbers which I can't understand...basically I am performing a permanova test using the adonis2 function, from the vegan package, and what I see is a very significant p-value with a very low f-statistic and I was wondering how this is possible.

This is what I have:

Permutation test for adonis under reduced model
Terms added sequentially (first to last)
Permutation: free
Number of permutations: 999

adonis2(formula = transposed_taxa ~ Treatment, data = adonis_meta, permutations = permuts, method = "bray", by = "terms")
Df SumOfSqs      R2     F Pr(>F)
Treatment  1   0.3506 0.08074 1.581  0.002 **
Residual  18   3.9912 0.91926
Total     19   4.3418 1.00000


I also tried an online F distribution calculator and, if I did the computations correctly (i.e., filled the proper number of degrees of freedom), the calculator gives me a non-significant p-value for the f-statistic I get from the test. Also, I tried the other way around, i.e., I provided the p-value and I got a very high f-statistics. However, I may be completely wrong and/or this calculator does not work for permanova test.

What am I missing here?

EDIT: this code provide the results i am seeing:

# load library
library("vegan")

# set seed
set.seed(131)

meta <- structure(list(treatment = structure(c(1L,
2L, 2L, 2L, 2L, 1L, 2L, 1L, 1L, 1L, 1L, 2L, 2L, 1L, 1L, 1L, 1L,
2L, 2L, 2L), levels = c("treat", "ctrl"), class = "factor"), SampleID = c("smpl_001",
"smpl_002", "smpl_003", "smpl_004", "smpl_005", "smpl_006", "smpl_007", "smpl_008",
"smpl_009", "smpl_010", "smpl_011", "smpl_012", "smpl_013", "smpl_014", "smpl_015",
"smpl_016", "smpl_017", "smpl_018", "smpl_019", "smpl_020")), row.names = c("smpl_001",
"smpl_002", "smpl_003", "smpl_004", "smpl_005", "smpl_006", "smpl_007", "smpl_008",
"smpl_009", "smpl_010", "smpl_011", "smpl_012", "smpl_013", "smpl_014", "smpl_015",
"smpl_016", "smpl_017", "smpl_018", "smpl_019", "smpl_020"), class = "data.frame")

# build distance matrix with bray curtis metric
dist_mat <- structure(c(0.724637600797927, 0.810151211442847, 0.677040935047858,
0.684990809226583, 0.773841077947218, 0.730965541771736, 0.721636429281786,
0.670062302186762, 0.735219755374301, 0.726420817041101, 0.713890852756624,
0.672481163496018, 0.770155244262034, 0.736284810905502, 0.725613337053673,
0.729027437249588, 0.684174177335026, 0.731597842534973, 0.744984703810319,
0.624016277905062, 0.716705596289758, 0.642368044368606, 0.660041089097984,
0.683430554390529, 0.637694222022575, 0.696747540033531, 0.678073194220625,
0.606862380345106, 0.57055508520052, 0.551073859282405, 0.685299854417393,
0.576578503650073, 0.70888030875411, 0.753358012196228, 0.559701576529528,
0.628139583570118, 0.616139151118536, 0.673897628859827, 0.688365913838432,
0.600801291804287, 0.653870227670454, 0.713403074119784, 0.622562333387564,
0.628100195801016, 0.638223638807926, 0.621583052281211, 0.639684018803079,
0.658541917209453, 0.62143045183579, 0.711481887009044, 0.753101948589314,
0.652118731984756, 0.669405500109936, 0.631233748354645, 0.681137745516244,
0.75990843296923, 0.704545253550062, 0.733485020719486, 0.70159943122252,
0.687665942056175, 0.71194111729798, 0.687556422198947, 0.672450715128387,
0.729058498800823, 0.777450338280718, 0.724047236746885, 0.693695657946689,
0.649024860632527, 0.743978052380142, 0.690306665100079, 0.66738824968164,
0.666456720997265, 0.736939480231912, 0.701398808286334, 0.666061916019718,
0.673314770321561, 0.678582532793769, 0.601722977395924, 0.694594001795866,
0.666715778630456, 0.733905602972688, 0.708928297345911, 0.660957906863241,
0.688380488856927, 0.66749857777161, 0.638562693219007, 0.709099622247953,
0.661582073250648, 0.633361258020319, 0.64395516014329, 0.649660296892919,
0.693266403675858, 0.619593981263679, 0.610426150945898, 0.680254103567928,
0.704351725342505, 0.655961485324122, 0.682236566670637, 0.60440942570617,
0.717559929520578, 0.678396363326736, 0.662793617648454, 0.657884773482801,
0.611508752657517, 0.695249293387878, 0.641413832130993, 0.631289777942896,
0.76873495257605, 0.7214801270069, 0.648605405437042, 0.638675714854417,
0.690476980816459, 0.624196684143274, 0.716828273978603, 0.620623296876388,
0.734305024372298, 0.696968360773382, 0.731531545670681, 0.590046390437714,
0.656457660565246, 0.767672873743218, 0.662583670754548, 0.741384141808377,
0.632153877463622, 0.686203048119573, 0.681849943901682, 0.679030715444911,
0.695081943961127, 0.670973436524669, 0.655707226050489, 0.592390953713221,
0.719296118412277, 0.681242617504174, 0.709011062976595, 0.67783231497305,
0.628507609160785, 0.698992117288368, 0.692043273990139, 0.635338402719872,
0.657103639492005, 0.716464174187625, 0.615559649585827, 0.695082938920572,
0.667734702508218, 0.616496859490065, 0.642779258886188, 0.653646025092285,
0.631693957514602, 0.611457903279869, 0.688517163734204, 0.737259708862597,
0.603583398496329, 0.656242818753164, 0.597835522574495, 0.594058163092095,
0.670135094778774, 0.621802926897363, 0.744151393706153, 0.723831207989047,
0.563513462547226, 0.628881872570669, 0.62794706034554, 0.676684493053249,
0.622600780389654, 0.698874934198522, 0.7223868940598, 0.552933805235236,
0.664090363014779, 0.611138112893922, 0.596969123444625, 0.73702614785897,
0.66350703692813, 0.648073310354135, 0.611886898022537, 0.698582583703285,
0.683497738003825, 0.753238008973899, 0.637095874502176, 0.625844086528647,
0.629241237689765, 0.732814733043977, 0.641962575290711, 0.769763636147583,
0.711868946785738, 0.701186509994886, 0.689603853801978, 0.766601551308245,
0.634042239970803, 0.624152094184768, 0.701782607184278), maxdist = 1, Size = 20L, Labels = c("smpl_001",
"smpl_002", "smpl_003", "smpl_004", "smpl_005", "smpl_006", "smpl_007", "smpl_008",
"smpl_009", "smpl_010", "smpl_011", "smpl_012", "smpl_013", "smpl_014", "smpl_015",
"smpl_016", "smpl_017", "smpl_018", "smpl_019", "smpl_020"), Diag = FALSE, Upper = FALSE, method = "bray", class = "dist")

# run test

• You apparently have quite a small dataset. Can you edit your post to include the data (by pasting in the output of dput(adonis_meta)) and the code you ran? Mar 28, 2023 at 11:13
• the dataset comprises 20 samples (10 for each treatment) and ~4.5k taxa since these are ecology data. so, yes it may be possible that the number of samples is small but i have no idea how big the average dataset looks like. and i am afraid i won't be able to upload the dataset.
– gabt
Mar 28, 2023 at 12:35
• you could post the dissimilarity matrix which should (i think?) be sufficient for this test. Mar 28, 2023 at 15:08
• @GeorgeSavva, yep, indeed it is. i did it, thank you!
– gabt
Mar 29, 2023 at 8:11

You can have a look at the permutation distribution of F using function permustats to extract permutation values (randomized values) and densityplot and other support functions to display those values, or densityplot(permustats(<yourmodel>)).
• ok, so the F statistics that i see in the result is this pseudo-F that is used to compute the p value using the formula mentioned in the wikipedia page (en.wikipedia.org/wiki/Permutational_analysis_of_variance). so, the pvalue is giving me the likelihood of observing a F statistic that is bigger (more extreme) than the one adonis2 is returning? what does this tells me about the data, though?