I am generating a GCP deployment manager template, which describes a big number of identical (100+) hosts, all of them having the same (large) user data.
In AW CloudFormation it is possible to use some sort of data reuse, where you can define a piece of data and just reference in many places, and so minimize the template size drastically.
Is it possible to have something like this in GCP? I didn't manage to find anything like that.
How am I supposed to solve this problem?
Deployment Manager supports jinja templates, so what you're looking for is using a variable (and possibly some loop):
<% set commonData="blah blah blah" %>
resources:
<% for i in range(100) %>
- type: mytype
name: host-no-{{ loop.index }}
properties:
some-property: {{commonData}}
<% endfor %>
It's an easy mechanism when you use gcloud cli as you can just call it and provide jinja file directly (--template instead of --config):
gcloud deployment-manager deployments create my-deployment --template template.jinja
Things get a bit more complicated when you create a deployment via API and cannot leverage cli utility for pulling and packaging your template(s).
Generally, a "proper way" to use DM is to have some template(s) (jinja or py) with variables, logic, loops and call them with a set of parameters (properties) from a static yaml file (config). DM API works the same way - you need to give it a yaml file (config) and optionally all templates (as imports). So the example above could become something like this:
config.yaml
-----------------------------
imports:
- template.jinja
resources:
- type: template.jinja
name: my-template
properties:
commonData: blah blah blah
.
template.jinja
-----------------------------
resources:
<% for i in range(100) %>
- type: mytype
name: host-no-{{ loop.index }}
properties:
some-property: {{ properties.commonData }}
<% endfor %>
Now all that is left is to uglify yaml into single lines and package into a json object matching deployment resource schema (https://cloud.google.com/deployment-manager/docs/reference/latest/deployments#resource). Below a working example of creating two subnets with common description:
{
"target": {
"config": {
"content": "imports: \n- blah.jinja\nresources: \n- type: blah.jinja\n name: test\n properties:\n description: blah blah blah\n names:\n - one\n - two"
},
"imports": [
{
"name": "blah.jinja",
"content": "resources: \n{% for name in properties.names %}\n- type: compute.v1.subnetwork\n name: test-{{name}}\n properties:\n region: europe-west6\n ipCidrRange: 192.168.{{loop.index}}.0/24\n description: {{properties.description}}\n network: default\n{% endfor %}"
}
]
},
"name": "too-many-blahs"
}