How to extract common `yq` to keep script DRY?

user_807037 0
Viewed 48

I have the following (modified from from https://stackoverflow.com/a/70152440/807037):

yq eval-all '
    .clusters = ( 
        (   
            (.clusters[] | {.name: .}) as $item ireduce ({}; . * $item)
        ) as $uniqueMap |
        ( $uniqueMap  | to_entries | .[]) as $item ireduce([]; . + $item.value)
    ) | 
    .contexts = ( 
        (
            (.contexts[] | {.name: .}) as $item ireduce ({}; . * $item)
        ) as $uniqueMap |
        ( $uniqueMap  | to_entries | .[]) as $item ireduce([]; . + $item.value)
    ) | 
    select(fi == 0)' konfig monfig

How can the following common code be extracted so as to keep the script DRY:

.«KEY» = (
        (
            (.«KEY»[] | {.name: .}) as $item ireduce ({}; . * $item)
        ) as $uniqueMap |
        ( $uniqueMap  | to_entries | .[]) as $item ireduce([]; . + $item.value)
    )

Input files:

# konfig
apiVersion: apiVersion-keep
clusters:
- cluster:
    certificate-authority-data: cad-0
    server: server-0
  name: name-0
- cluster:
    certificate-authority-data: cad-1-discard
    server: server-1-discard
  name: name-1
- cluster:
    certificate-authority-data: cad-2
    server: server-2
  name: name-2
contexts:
- context:
    cluster: cluster-0
    user: user-0
  name: name-0
- context:
    cluster: cluster-1-discard
    user: user-1-discard
  name: name-1
- context:
    cluster: cluster-2
    user: user-2
  name: name-2
current-context: name-keep

# monfig
apiVersion: apiVersion-discard
clusters:
- cluster:
    certificate-authority-data: cad-1-keep
    server: server-1-keep
  name: name-1
contexts:
- context:
    cluster: cluster-1-keep
    user: user-1-keep
  name: name-1
current-context: name-discard

Expected:

apiVersion: apiVersion-keep
clusters:
- cluster:
    certificate-authority-data: cad-0
    server: server-0
  name: name-0
- cluster:
    certificate-authority-data: cad-1-keep
    server: server-1-keep
  name: name-1
- cluster:
    certificate-authority-data: cad-2
    server: server-2
  name: name-2
contexts:
- context:
    cluster: cluster-0
    user: user-0
  name: name-0
- context:
    cluster: cluster-1-keep
    user: user-1-keep
  name: name-1
- context:
    cluster: cluster-2
    user: user-2
  name: name-2
current-context: name-keep
yq
2 Answers

The issue is a little trickier than it appears because a use case for |= is updating each matching left hand side node with respect to itself. .clusters results in two nodes (as with .contexts) and each of those nodes is updated independently. yq doesn't know to group the nodes together. After playing around a little I got this to work:

./yq eval-all '
  . ref $r |
  with( ("clusters", "contexts"); 
    $r[.] = ( 
        (
            ($r[.] | .[] | {.name: .}) as $item ireduce ({}; . * $item)
        ) as $uniqueMap |
        ( $uniqueMap  | to_entries | .[]) as $item ireduce([]; . + $item.value)
    )
  ) | select(fi==0)' file1.yaml file2.yaml

Explanation:

  • . ref $r Create a reference to the root context, called $r. This matches the top level nodes (file1 and file2).
  • Using the with operator, I can parameterise the merge expression against $r, passing in the two paths that need to be merged. Each path is run against the root context $r in $r[.].

Hope that makes sense!

Disclaimer: I wrote yq

user_11682230

Generally speaking (due to missing sample data), you can use the update operator |= (available since v4.3.0), enabling you to address by context . on the RHS. Then, as the absolute context only appears on the LHS, you can just list at once all contexts you want this to be applied to.

(.clusters, .contexts) |= ( .[] | ... )
user_21584790
Related