We need to provide a double-quote escaped JSON string into AWS secrets for a nodeJS product. Single-within-double quoted format will not work with the application. The structured data is in dict-list-dict format. Example:
{
"data": "[{\"URL\": \"beta.test.net\", \"token\": \"beta-token\"}, {\"URL\": \"beta-prod.test.net\", \"token\": \"beta-prod-token\"}]"
}
I tried a number of filters to get ansible and jinja to take the JSON variable and convert it to a string in the format wanted, but results are never quite right. | to_json | string gives the following result:
{
"data": "[{'URL': 'beta.test.net', 'token': 'beta-token'}, {'token': 'test-token', 'URL': 'beta-prod.test.net'}]"
}
And when I try using replace or regex replace, it appears that even with a guard character I end up hitting a python parsing error. Here's an example just using a replace with a grave's accent '`' escape.
TASK [Update secret in AWS] ***************************************************************************************************
task path: path:27
The full traceback is:
Traceback (most recent call last):
File "/home/.local/lib/python3.5/site-packages/ansible/executor/task_executor.py", line 146, in run
res = self._execute()
File "/home/.local/lib/python3.5/site-packages/ansible/executor/task_executor.py", line 587, in _execute
self._task.post_validate(templar=templar)
File "/home/.local/lib/python3.5/site-packages/ansible/playbook/task.py", line 296, in post_validate
super(Task, self).post_validate(templar)
File "/home/.local/lib/python3.5/site-packages/ansible/playbook/base.py", line 431, in post_validate
value = templar.template(getattr(self, name))
File "/home/.local/lib/python3.5/site-packages/ansible/template/__init__.py", line 623, in template
disable_lookups=disable_lookups,
File "/home/.local/lib/python3.5/site-packages/ansible/template/__init__.py", line 578, in template
disable_lookups=disable_lookups,
File "/home/.local/lib/python3.5/site-packages/ansible/template/__init__.py", line 806, in do_template
data = _escape_backslashes(data, myenv)
File "/home/.local/lib/python3.5/site-packages/ansible/template/__init__.py", line 149, in _escape_backslashes
for token in jinja_env.lex(d2):
File "/home/.local/lib/python3.5/site-packages/jinja2/lexer.py", line 739, in tokeniter
name, filename)
jinja2.exceptions.TemplateSyntaxError: unexpected char '`' at 51
line 1
fatal: [beta-prod]: FAILED! => {
"msg": "Unexpected failure during module execution.",
"stdout": ""
}
Using regex replace I can get all the way to removing the single quotes and replacing it with a '\', but any attempt to add '\"' will always fail with the above parsing error.
Here's the ansible code calling AWS secrets with a few of my attempts to provide the correct format:
- name: Update secret in AWS
aws_secret:
name: '{{ aws_secret_name }}'
state: present
secret_type: 'string'
# secret: "{{ token_list_update | to_json | replace(\"'\",\"\") | string }}"
secret: '{{ token_list_update | to_json | string }}'
# secret: ' {"data": "{{ token_list | to_json | regex_replace(`"`,`\\"`) }}"}'
Any help would be greatly appreciated. I'm getting the idea I made need to write my own filter to handle this. I did ask if the application would accept single-within-double-quoted JSON string, but the team indicated this was not acceptable. The format must be a double-quote escaped JSON string.
Final Edit:
After a few more days of work, figured this one out. At each step you need to cast the correct type to preserve JSON formatting. When doing updates each list element needs to be filtered to_json. Example:
token_list: "{{ (token_list | default([]) + [_update]) | to_json }}"
_update is a helper variable that is a JMESPath search
The final submission into AWS:
- name: Update secret in AWS
aws_secret:
name: "{{ aws_secret_name }}"
state: present
secret_type: "string"
secret: "{\"data\":{{ token_list | to_json}}}"
