rules_k8s - k8s_deploy template away images

user_6135889 0
Viewed 203

We got a project which consists of more than 20 small services that all reside inside the same repository and are built using bazel.

To reduce management overhead we would like to automagically generate as much as possible, including our images and k8s deployments. So the question is:

Is there a way to avoid setting the image key in the k8s_deploy step by a rule or function?

We already got a rule which is templating the image inside our manifest to have the image name (and k8s object name) based on the label:

_TEMPLATE = "//k8s:deploy.yaml"

def _template_manifest_impl(ctx):
  name = '{}'.format(ctx.label).replace("//cmd/", "").replace("/", "-").replace(":manifest", "")
  ctx.actions.expand_template(
    template = ctx.file._template,
    output = ctx.outputs.source_file,
    substitutions = {
      "{NAME}": name,
    },
  )

template_manifest = rule(
  implementation = _template_manifest_impl,
  attrs = {
    "_template": attr.label(
      default = Label(_TEMPLATE),
      allow_single_file = True,
    ),
  },
  outputs = {"source_file": "%{name}.yaml"},
)

This way the service under //cmd/endpoints/customer/log would result in the image eu.gcr.io/project/endpoints-customer-log.

While this works fine so far, we still have to manually set the images dict for k8s_deploy like this:

k8s_deploy(
    name = "dev",
    images = {
        "eu.gcr.io/project/endpoints-customer-log:dev": ":image",
    },
    template = ":manifest",
)

It would be great to get rid of this, but I failed to find a way yet. Using a rule does not work because images does not take a label and using a function does not work because i found no way of accessing context in there.

Am I missing something?

bazel kubernetes
1 Answers

The solution I found to get the container registry names out of the build step, was to use bazel for build and skaffold for deploy. Both steps are performed in the same CI pipeline.

My skaffold.yaml is very simple, and provides the mapping of bazel targets to gcr names.

apiVersion: skaffold/v2alpha4
kind: Config
metadata:
  name: my_services
build:
  tagPolicy:
    gitCommit: 
      variant: AbbrevCommitSha
  artifacts:
  - image: gcr.io/jumemo-dev/service1
    bazel:
      target: //server1/src/main/java/server1:server1.tar
  - image: gcr.io/jumemo-dev/service2
    bazel:
      target: //server2/src/main/java/server2:server2.tar

It is invoked using:

$ skaffold build
user_25159750
Related