Edit YAML file with Bash

Viewed 31856

I'm trying to edit the following YAML file

db:
  host: 'x.x.x.x.x'
  main:
    password: 'password_main'
  admin:
    password: 'password_admin'

To edit the host part, I got it working with

sed -i "/^\([[:space:]]*host: \).*/s//\1'$DNS_ENDPOINT'/" config.yml

But I can't find a way to update the password for main and admin (which are different values).

I tried to play around with \n and [[:space:]] and got different flavours of:

sed -i "/^\([[:space:]]*main:\n*[[:space:]]*password: \).*/s//\1'$DNS_ENDPOINT'/" config.yml

But never got it to work. Any help greatly appreciated!

Edit - Requirement: no external binaries/tools. Just good ol' bash.

5 Answers

Since you don't want to install yq you could use that you most probably already have installed.

Here are the fundamentals:

#!/usr/bin/python

import yaml

with open("config.yml") as f:
    y = yaml.safe_load(f)
    y['db']['admin']['password'] = 'new_admin_pass'
    print(yaml.dump(y, default_flow_style=False, sort_keys=False))

Output:

db:
  host: x.x.x.x.x
  main:
    password: password_main
  admin:
    password: new_admin_pass

A similar piece of code as a one-liner that you can put in a script would look something like this (and produce the same output):

python -c 'import yaml;f=open("config.yml");y=yaml.safe_load(f);y["db"]["admin"]["password"] = "new_admin_pass"; print(yaml.dump(y, default_flow_style=False, sort_keys=False))'

If you'd like to save the output to a file, you can provide an output stream as the second argument to dump():

#!/usr/bin/python

import yaml

with open("config.yml") as istream:
    ymldoc = yaml.safe_load(istream)
    ymldoc['db']['admin']['password'] = 'new_admin_pass'

with open("modified.yml", "w") as ostream:
    yaml.dump(ymldoc, ostream, default_flow_style=False, sort_keys=False)

If you'd like to overwrite the original file, I recommend writing to a temporary file first and only if that succeeds, use os.rename to move that file in place of the original one. That's to minimize the risk of creating a corrupt config.yml in case of problems.

Note: Using a YAML parser like yq (or yq) will be a way more reliable solution.


However, I've used the following 'technique' to alter a 'pre-defined' line though the help of and like so;

/tmp/config.yml

db:
  host: 'x.x.x.x.x'
  main:
    password: 'password_main'
  admin:
    password: 'password_admin'
  1. Get the line number where your 'old-password' is located:
    grep -n 'password_admin' /tmp/config.yml | cut -d ':' -f1
    

    6

  2. Then, use sed to override that line with your new password:
    sed -i '6s/.*/    password: \'new_admin_pass\'/' /tmp/config.yml
    

The new file now looks like this:

db:
  host: 'x.x.x.x.x'
  main:
    password: 'password_main'
  admin:
    password: 'new_admin_pass'

Note

  • Keep in mind that any special chars (&, \, /) in the password will cause sed to misbehave!

  • This could fail if the indent changes, since YAML cares about indentation. Just like I mentioned above, using a YAML parser will be a much more reliable solution!

This is by no way as reliable as yq but you can use this awk if your yaml file structure is same as how it is shown in question:

pw='new_&pass'
awk -v pw="${pw//&/\\\\&}" '/^[[:blank:]]*main:/ {
   print
   if (getline > 0 && $1 == "password:")
      sub(/\047[^\047]*\047/, "\047" pw "\047")
} 1' file
db:
  host: 'x.x.x.x.x'
  main:
    password: 'new_&pass'
  admin:
    password: 'password_admin'
$ awk -v new="'sumthin'" 'prev=="main:"{sub(/\047.*/,""); $0=$0 new} {prev=$1} 1' file
db:
  host: 'x.x.x.x.x'
  main:
    password: 'sumthin'
  admin:
    password: 'password_admin'

or if your new text can contain escape sequences that you don't want expanded (e.g. \t or \n), as seems likely when setting a password, then:

new="'sumthin'" awk 'prev=="main:"{sub(/\047.*/,""); $0=$0 ENVIRON["new"]} {prev=$1} 1' file

See How do I use shell variables in an awk script? for why/how I use ENVIRON[] to access a shell variable rather than setting an awk variable in that second script.

As mentioned by experts in other answers too, yq should be the proper way but in case someone doesn't have it then one could try following.

awk -v s1="'" -v new_pass="new_value_here" '
/main:/{
  main_found=1
  print
  next
}
main_found && /password/{
  next
}
/admin:/ && main_found{
  print "    password: " s1 new_pass s1 ORS $0
  main_found=""
  next
}
1
'  Input_file

NOTE: In case you want to save output into Input_file itself then append > temp && mv temp Input_file to above solution.

Related