how to specify the amount of RAM to use for a script in linux

user_11131402 0
Viewed 44

Let's say I want to edit all the files in a folder, changing every header. I'm using this script:

for thing in $(ls $1); do sed -i '1c\SNP A2 A1 beta N P' $thing done

The problem is that it takes a lot of time. So, I'd like to find a way to dedicate more RAM for this script, in order to do it quickly. Is it possible?

ram editing time linux
2 Answers

I'd like to find a way to dedicate more RAM for this script, in order to do it quickly. Is it possible?

No. The tools you use, bash to enumerate your files and sed to edit them, take the RAM they need to do their work. They can't use more RAM even if they had a way to give it to them.

You can run your sed operations in parallel. This uses more cores on your machine and may finish faster. Put an & character after each command in the loop. Something like this.

for thing in $(ls $1); do 
  sed -i '1c\SNP A2 A1 beta N P' $thing &
done
user_2056080

With GNU Parallel you can control how many are run in parallel:

doit() {
  sed -i '1c\SNP A2 A1 beta N P' "$1"
}
export -f doit
ls $1 | parallel -j5 doit

Adjust -j5 until you find the number that gives you most throughput.

user_3630280
Related