Splitting records in Apache Nifi

Question

I am working on a Jolt transforms processor in Apache Nifi, I am facing some issues, please help me out.

Input:

{
  "resourceid": "d6315d4d7f0c",
  "timestamp": [
    166406,
    166404,
    166504
  ],
  "Key": [
    "mem",
    "net",
    "diskspace"
  ],
  "data": [
    89,
    90,
    91
  ]
}

Expected output:

[
  {
    "resourceid": "d6315d4d7f0c",
    "timestamp": 166406,
    "Key": "mem",
    "data": 89
  },
   {
    "resourceid": "d6315d4d7f0c",
    "timestamp": 166404,
    "Key": "net",
    "data":  90
  },
   {
    "resourceid": "d6315d4d7f0c",
    "timestamp": 166504,
    "Key": "diskspace",
    "data": 91
  
  }
]

Barbaros Özhan · Answer

You can loop through the indexes of one of the arrays(in this case I've chosen timestamp) within a single shift transformation such as

[
  {
    "operation": "shift",
    "spec": {
      "timestamp": {
        "*": {
          "@(2,resourceid)": "[&].resourceid",
          "@": "[&].&2", // "[&]" represents indexes 0,1,2... nested within array to construct a structure of type array
          "@(2,Key[&])": "[&].Key", // go two levels up the three in order to reach the level of "Key", and grab its value
          "@(2,data[&])": "[&].data"
        }
      }
    }
  }
]

the demo on the site http://jolt-demo.appspot.com/ is

Splitting records in Apache Nifi

1 Answers