Applying cardinality for multiple columns in Jolt

Question

I am trying to apply Jolt for below data

input:

[
  {
    "id": "500",
    "code": "abc",
    "date": "2020-10-10",
    "category": 1,
    "amount": 100,
    "result": 0
  },
  {
    "id": "500",
    "code": "abc",
    "date": "2020-10-10",
    "category": 2,
    "amount": 200,
    "result": 1
  }
]

jolt used:

[
  {
    "operation": "shift",
    "spec": {
      "*": {
        "id": "@(1,id).id",
        "code": "@(1,id).code",
        "date": "@(1,id).group1.date",
        "category": "@(1,id).group1.group2[&1].category"
      }
    }
  },
  {
    "operation": "cardinality",
    "spec": {
      "*": {
        "id": "ONE"
      }
    }
  },
  {
    "operation": "shift",
    "spec": {
      "*": ""
    }
  }
]

current output:

{
  "id": "500",
  "code": [
    "abc",
    "abc"
  ],
  "group1": {
    "date": [
      "2020-10-10",
      "2020-10-10"
    ],
    "group2": [
      {
        "category": 1
      },
      {
        "category": 2
      }
    ]
  }
}

expected:

{
  "id": "500",
  "code": "abc",
  "group1": {
    "date": "2020-10-10",
    "group2": [
      {
        "category": 1
      },
      {
        "category": 2
      }
    ]
  }
}

If i keep column of code & date in cardinality, it's fine. But in my use case, there are multiple such columns to be added. Are there any better ways to handle this scenario?

user_5841306 · Answer

You should add each added node and use "*" wildcard to represent the rest of the attributes within the cardinality transformation such as

{
  "operation": "cardinality",
  "spec": {
    "*": {
      "*": "ONE",
      "group1": {
        "*": "ONE",
        "group2": "MANY"
      }
    }
  }
}

where "group2": "MANY" will make group2 to be excepted for extracting only the first element of the respective list.

the demo on the site http://jolt-demo.appspot.com/ :

Applying cardinality for multiple columns in Jolt

1 Answers