NAs when attempting to get standard error

user_13111957 0
Viewed 136

R is doing a very strange thing where it is not giving me an error message and instead just not computing what I've told it to compute. I'm attempting to find the standard error of a variable and the command is producing NAs instead and I cannot figure out why. Here's my code for getting the mean and standard error:

ReHo_mean_Esc_1 <- ReHo_Group_Esc_1 %>% group_by(Group) %>% summarise(Value=mean(Value), se=sd(Value)/sqrt(n()))

My variable of interest is called Value. Here's my dataframe:

ID  Clu Group Value Esc Nal 
422 1   LgA 3.26090 94  7.50
501 1   LgA 3.32376 139 15.25
503 1   LgA 2.76855 24  31.50
521 1   LgA 1.81475 -28 6.75
522 1   LgA 1.80966 58  13.00
523 1   LgA 3.97502 76  10.25
603 1   LgA 1.78573 76  18.00
604 1   LgA 3.70577 54  10.00
605 1   LgA 2.93304 51  18.00
613 1   LgA 3.68118 116 17.00
429 1   ShA 2.61634 -33 5.75
430 1   ShA 3.39848 13  12.75
431 1   ShA 3.40785 -33 9.75
432 1   ShA 4.38024 50  4.75
513 1   ShA 4.14605 8   10.50
514 1   ShA 3.86332 0   10.75
518 1   ShA 2.96312 0   13.00
519 1   ShA 2.82937 -33 7.50
610 1   ShA 5.07850 13  26.00
612 1   ShA 4.14895 56  4.00
614 1   ShA 3.83926 42  8.25

My summarize command has no issues producing the mean for each group but it gives me NAs for the standard error and I have no idea why. Any ideas?

Thanks!

standard-error r
3 Answers

Don't name your new variable Value. dplyr is different to base R in that it allows newly created variables to be immediately available within the same function.

ReHo_Group_Esc_1 %>% 
  group_by(Group) %>% 
  summarise(mValue=mean(Value), se=sd(Value)/sqrt(n()))
# A tibble: 2 x 3
  Group mValue    se
  <chr>  <dbl> <dbl>
1 LgA     2.91 0.266
2 ShA     3.70 0.223
user_75145270

The issue is that by the time you calculate sd(Value), the Value column of length 21 has been converted into a column of length 1 (per group). Two clues:

  1. sd of anything length 1 is NA;

  2. Try to replace sd with length, and you'll see that it's getting just one value (errr, Value :-) (this is a play on @CalumYou's comment):

    ReHo_Group_Esc_1 %>%
  group_by(Group) %>%
  summarise(Value=mean(Value), se=length(Value))
# # A tibble: 2 x 3
#   Group Value    se
#   <chr> <dbl> <int>
# 1 LgA    2.91     1
# 2 ShA    3.70     1

    whereas if you swap the order of calculations, you'll see something different:

    ReHo_Group_Esc_1 %>%
  group_by(Group) %>%
  summarise(se=length(Value), Value=mean(Value))
# # A tibble: 2 x 3
#   Group    se Value
#   <chr> <int> <dbl>
# 1 LgA      10  2.91
# 2 ShA      11  3.70

Try calculating sd first:

ReHo_Group_Esc_1 %>%
  group_by(Group) %>%
  summarise(
    se = sd(Value)/sqrt(n()),
    Value = mean(Value)
  )
# # A tibble: 2 x 3
#   Group    se Value
#   <chr> <dbl> <dbl>
# 1 LgA   0.266  2.91
# 2 ShA   0.223  3.70
user_33582720

You can try:

    ReHo_Group_Esc_1 %>% group_by(Group) %>% 
    summarise(Value=mean(Value,na.rm=T), se=sd(Value,na.rm=T)/sqrt(n()))
user_20808480
Related