ggpubr's compare_means and base R's pairwise.t.test give different results

user_13498996 0
Viewed 821

First time posting to stackoverflow, I hope somebody can help me. Thanks in advance!

I wanted to use the R package ggpubr to create a bar graph showing expression of a gene in different treatment groups, but I noticed that the included function compare_means or stat_compare_means returns way higher p-values for the comparison of all the groups than the R base function pairwise.t.test. Actually some values are way higher and some way lower. Does the ggpubr function use some more conservative assumption? Here is my data and the code sample:

   Target.Name Group     CT   dCT   f.change
81        Gen1   300 23.911 1.900 0.26794337
82        Gen1   300 24.990 3.190 0.10957572
83        Gen1   300 24.504 2.646 0.15965172
84        Gen1    30 26.379 4.486 0.04462512
85        Gen1    30 26.576 4.366 0.04852930
86        Gen1    30 27.154 4.912 0.03321549
87        Gen1     3 27.317 4.923 0.03298605
88        Gen1     3 27.119 5.288 0.02559490
89        Gen1     3 27.313 5.691 0.01935701
90        Gen1   0.3 27.388 5.857 0.01725311
91        Gen1   0.3 26.911 5.104 0.02909671
92        Gen1   0.3 26.872 5.816 0.01773816
93        Gen1     0 26.371 5.502 0.02206648
94        Gen1     0 27.283 5.778 0.01822421
95        Gen1     0 27.168 5.618 0.02034757

#-----------------------------------------
compare_means(dat_subset, formula = f.change ~ Group, method = "t.test")
pairwise.t.test(dat_subset$f.change, dat_subset$Group)

And the output is

> compare_means(dat_subset, formula = f.change ~ Group, method = "t.test")
# A tibble: 10 x 8
   .y.      group1 group2      p p.adj p.format p.signif method
   <chr>    <chr>  <chr>   <dbl> <dbl> <chr>    <chr>    <chr> 
 1 f.change 0      0.3    0.799   0.9  0.799    ns       T-test
 2 f.change 0      3      0.278   0.83 0.278    ns       T-test
 3 f.change 0      30     0.0351  0.32 0.035    *        T-test
 4 f.change 0      300    0.0767  0.54 0.077    ns       T-test
 5 f.change 0.3    3      0.450   0.9  0.450    ns       T-test
 6 f.change 0.3    30     0.0271  0.27 0.027    *        T-test
 7 f.change 0.3    300    0.0767  0.54 0.077    ns       T-test
 8 f.change 3      30     0.0573  0.46 0.057    ns       T-test
 9 f.change 3      300    0.0809  0.54 0.081    ns       T-test
10 f.change 30     300    0.0980  0.54 0.098    ns       T-test
> pairwise.t.test(dat_subset$f.change, dat_subset$Group)

    Pairwise comparisons using t tests with pooled SD 

data:  dat_subset$f.change and dat_subset$Group 

    0      0.3    3      30    
0.3 1.0000 -      -      -     
3   1.0000 1.0000 -      -     
30  1.0000 1.0000 1.0000 -     
300 0.0034 0.0034 0.0036 0.0071

P value adjustment method: holm
ggpubr statistics r
2 Answers

To obtain the same results, you'll have to specify that you don't want the variances to be pooled (pool.sd=FALSE), since the default for pairwise.t.test is TRUE, but the default for compare_means is FALSE. (or vice-versa)

pairwise.t.test(x=dat_subset$f.change, g=dat_subset$Group, pool.sd = FALSE)

data:  dat_subset$f.change and dat_subset$Group 

    0    0.3  3    30  
0.3 0.90 -    -    -   
3   0.83 0.90 -    -   
30  0.32 0.27 0.46 -   
300 0.54 0.54 0.54 0.54

compare_means(dat_subset, formula = f.change ~ Group, method = "t.test")
# A tibble: 10 x 8
   .y.      group1 group2      p p.adj p.format p.signif method
   <chr>    <chr>  <chr>   <dbl> <dbl> <chr>    <chr>    <chr> 
 1 f.change 300    30     0.0980  0.54 0.098    ns       T-test
 2 f.change 300    3      0.0809  0.54 0.081    ns       T-test
 3 f.change 300    0.3    0.0767  0.54 0.077    ns       T-test
 4 f.change 300    0      0.0767  0.54 0.077    ns       T-test
 5 f.change 30     3      0.0573  0.46 0.057    ns       T-test
 6 f.change 30     0.3    0.0271  0.27 0.027    *        T-test
 7 f.change 30     0      0.0351  0.32 0.035    *        T-test
 8 f.change 3      0.3    0.450   0.9  0.450    ns       T-test
 9 f.change 3      0      0.278   0.83 0.278    ns       T-test
10 f.change 0.3    0      0.799   0.9  0.799    ns       T-test
user_75145270

Well they both claim to use holm as the default p.adjust but they seem to differ in whether they assume equal variance. Don't have enough of your data to truly test my hypothesis but they will yield different results per this example basically taken from the help file...

data("ToothGrowth")
df <- ToothGrowth
ggpubr::compare_means(len ~ supp, df, method = "t.test")
#> # A tibble: 1 x 8
#>   .y.   group1 group2      p p.adj p.format p.signif method
#>   <chr> <chr>  <chr>   <dbl> <dbl> <chr>    <chr>    <chr> 
#> 1 len   OJ     VC     0.0606 0.061 0.061    ns       T-test
ggpubr::compare_means(len ~ supp, df, method = "t.test", var.equal = TRUE)
#> # A tibble: 1 x 8
#>   .y.   group1 group2      p p.adj p.format p.signif method
#>   <chr> <chr>  <chr>   <dbl> <dbl> <chr>    <chr>    <chr> 
#> 1 len   OJ     VC     0.0604  0.06 0.06     ns       T-test
pairwise.t.test(df$len, df$supp)
#> 
#>  Pairwise comparisons using t tests with pooled SD 
#> 
#> data:  df$len and df$supp 
#> 
#>    OJ  
#> VC 0.06
#> 
#> P value adjustment method: holm
pairwise.t.test(df$len, df$supp, pool.sd = FALSE)
#> 
#>  Pairwise comparisons using t tests with non-pooled SD 
#> 
#> data:  df$len and df$supp 
#> 
#>    OJ   
#> VC 0.061
#> 
#> P value adjustment method: holm

Created on 2020-05-08 by the reprex package (v0.3.0)

user_91900340
Related