How to gsub through a list of names in a loop

user_9904275 0
Viewed 147

I have a batch of samples that I want to submit for processing on my university cluster. I have over 1000 samples that I need to run. Instead of having to manually create the scripts, I was wondering I could make a for-loop to substitute the samples ids. Each script is essentially the same, I just need to change out the sample ID and the location of the file.

df <- structure(list(V1 = c("#!/bin/bash", "#BSUB -W 1440", "#BSUB -n 16", 
                            "#BSUB -x", "#BSUB -R \"rusage[mem=4000] span[hosts=1]\"", "#BSUB -o /gpfs_common/share01/files/abc123.out.%J.txt", 
                            "#BSUB -e /gpfs_common/share01/files/abc123.err.%J.txt", "", 
                            "", "", "", "mcli cp def456/abc123 /panfs/roc/groups/0/location/data.base", 
                            "gzip /panfs/roc/groups/0/location/data.base/abc123", "mcli mv /panfs/roc/groups/0/location/data.base/abc123.gz def456/", 
                            "", "", "#BSUB -J abc123", "\t\t\t", "", "", "", "", "")), row.names = c(NA, 
                                                                                                     -23L), class = c("data.table", "data.frame"))

names <- list(V1 = c("D00268.merged.dedup.realn.haplotypecaller.g.vcf", 
                         "D00316.merged.dedup.realn.haplotypecaller.g.vcf", "D00426.merged.dedup.realn.haplotypecaller.g.vcf", 
                         "D00432.merged.dedup.realn.haplotypecaller.g.vcf", "D00474.merged.dedup.realn.haplotypecaller.g.vcf", 
                         "D00510.merged.dedup.realn.haplotypecaller.g.vcf", "D00574.merged.dedup.realn.haplotypecaller.g.vcf", 
                         "D00607.merged.dedup.realn.haplotypecaller.g.vcf", "D00619.merged.dedup.realn.haplotypecaller.g.vcf", 
                         "D00662.merged.dedup.realn.haplotypecaller.g.vcf"))
    
locations <- list(V1 = c("s3/lab/wgs/yrkt/D00268/gvcf/", "s3/lab/wgs/dach/D00316/gvcf/", 
                         "s3/lab/wgs/mnpd/D00426/gvcf/", "s3/lab/wgs/yrkt/D00432/gvcf/", 
                         "s3/lab/wgs/ckcs/D00474/gvcf/", "s3/lab/wgs/lbrt/D00510/gvcf/", 
                         "s3/lab/wgs/shlt/D00574/gvcf/", "s3/lab/wgs/shlt/D00607/gvcf/", 
                         "s3/lab/wgs/mnsc/D00619/gvcf/", "s3/lab/wgs/gtdn/D00662/gvcf/"
))

So df is just a master script that I wanted to run the for loop through. I changed the sample name to "abc123" and the sample location to "def456" in the master script so I can use something like gsub that can recognize those two patterns and subsitute them out for the sample ID and sample location. I was hoping to create a text file that looked something like this when I am done.

#!/bin/bash
#BSUB -W 1440
#BSUB -n 16
#BSUB -x
#BSUB -R "rusage[mem=4000] span[hosts=1]"
#BSUB -o /gpfs_common/share01/files/D00268.merged.dedup.realn.haplotypecaller.g.vcf.out.%J.txt
#BSUB -e /gpfs_common/share01/files/D00268.merged.dedup.realn.haplotypecaller.g.vcf.err.%J.txt




mcli cp s3/lab/wgs/yrkt/D00268/gvcf/D00268.merged.dedup.realn.haplotypecaller.g.vcf /panfs/roc/groups/0/location/data.base
gzip /panfs/roc/groups/0/location/data.base/D00268.merged.dedup.realn.haplotypecaller.g.vcf
mcli mv /panfs/roc/groups/0/location/data.base/D00268.merged.dedup.realn.haplotypecaller.g.vcf.gz s3/lab/wgs/yrkt/D00268/gvcf/


#BSUB -J D00268.merged.dedup.realn.haplotypecaller.g.vcf

I was thinking a for loop would be the easiest thing to do here but I am open to suggestions. Hopefully, this all makes sense. Let me know if you have any questions

I have used this for loop in the past but I have never used a for loop to gsub through a list

for(i in 1:nrow(df)){
  df[i,'V1'] <- gsub("abc123", "D00268.merged.dedup.realn.haplotypecaller.g.vcf", df[i,'V1'])
  df[i,'V1'] <- gsub("def456", "s3/lab/wgs/yrkt/D00268/gvcf/", df[i,'V1'])
  
}
r for-loop
2 Answers

To stick with the for-loop idea and modify your suggested approach, you can do something like:

for(i in 1:length(locations[[1]])){

df2 <- df
df2[,'V1'] <- gsub("abc123", names[['V1']][i], df2[,'V1'])
df2[,'V1'] <- gsub("def456", locations[['V1']][i], df2[,'V1'])
fileConn<-file(paste0("script_", i, ".sh" ))
writeLines(df2$V1, fileConn)
close(fileConn)

}
user_75167810

purrr::map2 can apply a function over the elements of two vectors.

res <- purrr::map2(
  names$V1,
  locations$V1, 
  function(name, location) {
    result <- gsub("abc123", name, df$V1)
    result <- gsub("def456", location, result)
    result
  }
)

length(res)
#> [1] 10

Here's the first result

cat(paste0(res[[1]], collapse = "\n"))
#> #!/bin/bash
#> #BSUB -W 1440
#> #BSUB -n 16
#> #BSUB -x
#> #BSUB -R "rusage[mem=4000] span[hosts=1]"
#> #BSUB -o /gpfs_common/share01/files/D00268.merged.dedup.realn.haplotypecaller.g.vcf.out.%J.txt
#> #BSUB -e /gpfs_common/share01/files/D00268.merged.dedup.realn.haplotypecaller.g.vcf.err.%J.txt
#> 
#> 
#> 
#> 
#> mcli cp s3/lab/wgs/yrkt/D00268/gvcf//D00268.merged.dedup.realn.haplotypecaller.g.vcf /panfs/roc/groups/0/location/data.base
#> gzip /panfs/roc/groups/0/location/data.base/D00268.merged.dedup.realn.haplotypecaller.g.vcf
#> mcli mv /panfs/roc/groups/0/location/data.base/D00268.merged.dedup.realn.haplotypecaller.g.vcf.gz s3/lab/wgs/yrkt/D00268/gvcf//
#>   
#>   
#>   #BSUB -J D00268.merged.dedup.realn.haplotypecaller.g.vcf
user_17519610
Related