How can I trim leading and trailing white space?

user_258334 0
Viewed 353919

I am having some trouble with leading and trailing white space in a data.frame.

For example, I look at a specific row in a data.frame based on a certain condition:

> myDummy[myDummy$country == c("Austria"),c(1,2,3:7,19)] 



[1] codeHelper     country        dummyLI    dummyLMI       dummyUMI       

[6] dummyHInonOECD dummyHIOECD    dummyOECD      

<0 rows> (or 0-length row.names)

I was wondering why I didn't get the expected output since the country Austria obviously existed in my data.frame. After looking through my code history and trying to figure out what went wrong I tried:

> myDummy[myDummy$country == c("Austria "),c(1,2,3:7,19)]
   codeHelper  country dummyLI dummyLMI dummyUMI dummyHInonOECD dummyHIOECD
18        AUT Austria        0        0        0              0           1
   dummyOECD
18         1

All I have changed in the command is an additional white space after Austria.

Further annoying problems obviously arise. For example, when I like to merge two frames based on the country column. One data.frame uses "Austria " while the other frame has "Austria". The matching doesn't work.

  1. Is there a nice way to 'show' the white space on my screen so that I am aware of the problem?
  2. And can I remove the leading and trailing white space in R?

So far I used to write a simple Perl script which removes the whites pace, but it would be nice if I can somehow do it inside R.

removing-whitespace r-faq trim whitespace r
15 Answers

Probably the best way is to handle the trailing white spaces when you read your data file. If you use read.csv or read.table you can set the parameterstrip.white=TRUE.

If you want to clean strings afterwards you could use one of these functions:

# Returns string without leading white space
trim.leading <- function (x)  sub("^\\s+", "", x)

# Returns string without trailing white space
trim.trailing <- function (x) sub("\\s+$", "", x)

# Returns string without leading or trailing white space
trim <- function (x) gsub("^\\s+|\\s+$", "", x)

To use one of these functions on myDummy$country:

 myDummy$country <- trim(myDummy$country)

To 'show' the white space you could use:

 paste(myDummy$country)

which will show you the strings surrounded by quotation marks (") making white spaces easier to spot.

f3lix_606170

Ad 1) To see white spaces you could directly call print.data.frame with modified arguments:

print(head(iris), quote=TRUE)
#   Sepal.Length Sepal.Width Petal.Length Petal.Width  Species
# 1        "5.1"       "3.5"        "1.4"       "0.2" "setosa"
# 2        "4.9"       "3.0"        "1.4"       "0.2" "setosa"
# 3        "4.7"       "3.2"        "1.3"       "0.2" "setosa"
# 4        "4.6"       "3.1"        "1.5"       "0.2" "setosa"
# 5        "5.0"       "3.6"        "1.4"       "0.2" "setosa"
# 6        "5.4"       "3.9"        "1.7"       "0.4" "setosa"

See also ?print.data.frame for other options.

user_1687470

Use grep or grepl to find observations with white spaces and sub to get rid of them.

names<-c("Ganga Din\t", "Shyam Lal", "Bulbul ")
grep("[[:space:]]+$", names)
[1] 1 3
grepl("[[:space:]]+$", names)
[1]  TRUE FALSE  TRUE
sub("[[:space:]]+$", "", names)
[1] "Ganga Din" "Shyam Lal" "Bulbul"
user_2125930

Use dplyr/tidyverse mutate_all with str_trim to trim the entire data frame:

myDummy %>%
  mutate_all(str_trim)

library(tidyverse)
set.seed(335)
df <- mtcars %>%
        rownames_to_column("car") %>%
        mutate(car = ifelse(runif(nrow(mtcars)) > 0.4, car, paste0(car, " "))) %>%
        select(car, mpg)

print(head(df), quote = T)
#>                    car    mpg
#> 1         "Mazda RX4 " "21.0"
#> 2      "Mazda RX4 Wag" "21.0"
#> 3        "Datsun 710 " "22.8"
#> 4    "Hornet 4 Drive " "21.4"
#> 5 "Hornet Sportabout " "18.7"
#> 6           "Valiant " "18.1"

df_trim <- df %>%
  mutate_all(str_trim)

print(head(df_trim), quote = T)  
#>                   car    mpg
#> 1         "Mazda RX4"   "21"
#> 2     "Mazda RX4 Wag"   "21"
#> 3        "Datsun 710" "22.8"
#> 4    "Hornet 4 Drive" "21.4"
#> 5 "Hornet Sportabout" "18.7"
#> 6           "Valiant" "18.1"

Created on 2021-05-07 by the reprex package (v0.3.0)

user_13703030

The best method is trimws().

The following code will apply this function to the entire dataframe.

mydataframe<- data.frame(lapply(mydataframe, trimws),stringsAsFactors = FALSE)
user_66051030

I tried trim(). It works well with white spaces as well as the '\n'.

x = '\n              Harden, J.\n              '

trim(x)
user_97030640

For whom it may be of interest, here a simple benchmarking. This is of course not capturing all weird cases, but so far we are still lacking the example where str_trim removes whitespace and trimws doesn't (see Richard Telford's comment to this answer). Doesn't seem to matter - the gsub option seems to be fastest :)

x <- c(" lead", "trail ", " both ", " both and middle ", " _special")
gsub_trim <- function (x) gsub("^\\s+|\\s+$", "", x)
res <- microbenchmark::microbenchmark(
  gsub = gsub_trim(x),
  trimws = trimws(x),
  str_trim = stringr::str_trim(x),
  times = 10^5
)
res
#> Unit: microseconds
#>      expr    min     lq      mean median       uq       max neval cld
#>      gsub 20.201 22.788  31.43943 24.654  28.4115  5303.741 1e+05 a  
#>    trimws 38.204 41.980  61.92218 44.420  51.1810 40363.860 1e+05  b 
#>  str_trim 88.672 92.347 116.59186 94.542 105.2800 13618.673 1e+05   c
ggplot2::autoplot(res)


sessionInfo()
#> R version 4.0.3 (2020-10-10)
#> Platform: x86_64-apple-darwin17.0 (64-bit)
#> Running under: macOS Big Sur 10.16
#> 
#> locale:
#> [1] en_GB.UTF-8/en_GB.UTF-8/en_GB.UTF-8/C/en_GB.UTF-8/en_GB.UTF-8
#> 
#> attached base packages:
#> [1] stats     graphics  grDevices utils     datasets  methods   base     
#> 
#> loaded via a namespace (and not attached):
#>  stringr_1.4.0
user_79411880
Related