Get a YAML file with HTTP and use it as a variable in an Ansible playbook

Viewed 1067

Background

I have a YAML file like this on a web server. I am trying to read it and make user accounts in the file with an Ansible playbook.

users:
  - number: 20210001
    name: Aoki Alice
    id: alice
  - number: 20210002
    name: Bob Bryant
    id: bob
  - number: 20210003
    name: Charlie Cox
    id: charlie

What I tried

To confirm how to read a downloaded YAML file dynamically with include_vars, I had written a playbook like this:

- name: Add users from list
  hosts: workstation
  tasks:
    - name: Download yaml
      get_url:
        url: http://fqdn.of.webserver/path/to/yaml.yml
        dest: "/tmp/tmp.yml"
      notify:
        - Read yaml
        - List usernames

  handlers:
    - name: Read yaml
      include_vars:
        file: /tmp/tmp.yml
        name: userlist

    - name: List usernames
      debug:
        var: "{{ item }}"
      loop: "{{ userlist.users }}"

Problem

In the handler Read yaml, I got the following error message. On the target machine (workstation.example.com), /tmp/tmp.yml is downloaded correctly.

RUNNING HANDLER [Read yaml] *****
fatal: [workstation.example.com]: FAILED! => {"ansible facts": {"userlist": []},
"ansible included var files": [], "changed": false, "message": "Could not find o
r access '/tmp/tmp. yml' on the Ansible Controller.\nIf you are using a module a
nd expect the file to exist on the remote, see the remote src option"}

Question

How can I get a YAML file with HTTP and use it as a variable with include_vars?

3 Answers

Another option would be to use the uri module to retrieve the value into an Ansible variable, then the from_yaml filter to parse it.

Something like:

- name: Add users from list
  hosts: workstation
  tasks:
    - name: Download YAML userlist
      uri:
        url: http://fqdn.of.webserver/path/to/yaml.yml
        return_content: yes
      register: downloaded_yaml
    - name: Decode YAML userlist
      set_fact:
        userlist: "{{ downloaded_yaml.content | from_yaml }}"

Note that uri works on the Ansible Controller, while get_url works on the target host (or on the host specified in delegate_to); depending on your network configuration, you may need to use different proxy settings or firewall rules to permit the download.

The include_vars task looks for files on the local (control) host, but you've downloaded the file to /tmp/tmp.yml on the remote host. There are a number of ways of getting this to work.

Perhaps the easiest is just running the download task on the control machine instead (note the use of delegate_to):

  tasks:
    - name: Download yaml
      delegate_to: localhost
      get_url:
        url: http://fqdn.of.webserver/path/to/yaml.yml
        dest: "/tmp/tmp.yml"
      notify:
        - Read yaml
        - List usernames

This will download the file to /tmp/tmp.yml on the local system, where it will be available to include_vars. For example, if I run this playbook (which grabs YAML content from an example gist I just created)...

- hosts: target
  gather_facts: false
  tasks:
    - name: Download yaml
      delegate_to: localhost
      get_url:
        url: https://gist.githubusercontent.com/larsks/70d8ac27399cb51fde150902482acf2e/raw/676a1d17bcfc01b1a947f7f87e807125df5910c1/example.yaml
        dest: "/tmp/tmp.yml"
      notify:
        - Read yaml
        - List usernames

  handlers:
    - name: Read yaml
      include_vars:
        file: /tmp/tmp.yml
        name: userlist

    - name: List usernames
      debug:
        var: item
      loop: "{{ userlist.users }}"

...it produces the following output:

RUNNING HANDLER [Read yaml] ******************************************************************
ok: [target]

RUNNING HANDLER [List usernames] *************************************************************
ok: [target] => (item=bob) => {
    "ansible_loop_var": "item",
    "item": "bob"
}
ok: [target] => (item=alice) => {
    "ansible_loop_var": "item",
    "item": "alice"
}
ok: [target] => (item=mallory) => {
    "ansible_loop_var": "item",
    "item": "mallory"
}

Side note: based on what I see in your playbook, I'm not sure you want to be using notify and handlers here. If you run your playbook a second time, nothing will happen because the file /tmp/tmp.yml already exists, so the handlers won't get called.

With @Larsks 's answer, I made this playbook that works correctly in my environment:

- name: Download users list
  hosts: 127.0.0.1
  connection: local
  become: no
  tasks:
  - name: Download yaml
    get_url:
      url: http://fqdn.of.webserver/path/to/yaml/users.yml
      dest: ./users.yml

- name: Add users from list
  hosts: workstation
  tasks:
  - name: Read yaml
    include_vars:
      file: users.yml
  - name: List usernames
    debug:
      msg: "{{ item.id }}"
    loop: "{{ users }}"

Point

Run get_url on the control host

As @Larsks said, you have to run the get_url module on the control host rather than the target host.

Add become: no to the task run on the control host

Without "become: no", you will get the following error message:

TASK [Gathering Facts] ******************************************************
fatal: [127.0.0.1]: FAILED! => {"ansible_facts": {}, "changed": false, "msg":
 "The following modules failed to execute: setup\n setup: MODULE FAILURE\nSee
 stdout/stderr for the exact error\n"}

Use connection: local rather than local_action

If you use local_action rather than connection: local like this:

- name: test get_url
  hosts: workstation
  tasks:
  - name: Download yaml
    local_action:
      module: get_url
      url: http://fqdn.of.webserver/path/to/yaml/users.yml
      dest: ./users.yml
  - name: Read yaml
    include_vars:
      file: users.yml
  - name: output remote yaml
    debug:
      msg: "{{ item.id }}"
    loop: "{{ users }}"

You will get the following error message:

TASK [Download yaml] ********************************************************
fatal: [workstation.example.com]: FAILED! => {"changed": false, "module_stde
rr": "sudo: a password is required\n", "module_stdout":"", "msg":"MODULE FAIL
URE\nSee stdout/stderr for the exact error", "rc": 1}

get_url stores a file on the control host

In this situation, the get_url module stores users.yml on the control host (in the current directory). So you have to delete the users.yml if you don't want to leave it.

Related