Replace a property in arbitrary data structure

Viewed 133

Let's say I have an array of arbitrary objects I don't know structure of. I would like to process it in a way that when a property matching some criteria in terms of property name is found at any nesting level, there is some action (mutation) done on it.

As an example, "find all properties with exact name" or "find all properties with a hyphen" and "replace the value with {redacted: true}".

I tried with R.set and R.lensProp but it seems to act on properties at the root level only. Here's my playground. Let's say I'd like to replace baz value with {redacted: true} or run R.map on it when it's an array.

const arr = [
  {
    id: 5,
    name: "foo",
    baz: [
      {
        a: 1,
        b: 2
      },
      {
        a: 10,
        b: 5
      }
    ],
    other: {
      very: {
        nested: {
          baz: [
            {
              a: 1,
              b: 2
            }
          ]
        }
      }
    }
  },
  {
    id: 6,
    name: "bar",
    baz: []
  }
];

const result = R.set(
  R.lensProp("baz"),
  {
    replaced: true,
  },
  arr[0]
);
console.log(result);
<script src="https://cdnjs.cloudflare.com/ajax/libs/ramda/0.27.1/ramda.min.js" integrity="sha512-rZHvUXcc1zWKsxm7rJ8lVQuIr1oOmm7cShlvpV0gWf0RvbcJN6x96al/Rp2L2BI4a4ZkT2/YfVe/8YvB2UHzQw==" crossorigin="anonymous"></script>

I feel I'm missing something very basic.

4 Answers

You'll need a recursive function that handles:

  • Arrays - mapping and calling itself on each item
  • Other objects - converting the object to array with R.toPairs, mapping the array of pairs, calling the predicate on each [key, value] pair, and calling itself on each value which is also an array
  • Primitives returning as is

const { curry, cond, is, pipe, toPairs, map, when, last, evolve, identity, fromPairs, T } = R;

const transform = curry((pred, arr) => cond([
  [is(Array), map(a => transform(pred, a))], // handle array - map each item
  [is(Object), pipe( // handle objects which are not arrays
    toPairs, // convert to pairs
    map(pipe( // map each pair
      pred, // call predicate on the pair
      when(pipe(last, is(Object)), evolve([ // handle array values
        identity, // return key as is
        a => transform(pred, a) // transform the array
      ])),
    )),
    fromPairs // convert back to an array
  )],
  [T, identity], // handle primitives
])(arr))

const redactBaz = transform(
  R.when(R.pipe(R.head, R.equals('baz')), R.always(['redacted', true]))
);

const arr = [{"id":5,"name":"foo","baz":[{"a":1,"b":2},{"a":10,"b":5}],"other":{"very":{"nested":{"baz":[{"a":1,"b":2}]}}}},{"id":6,"name":"bar","baz":[]}];

const result = redactBaz(arr);

console.log(result);
<script src="https://cdnjs.cloudflare.com/ajax/libs/ramda/0.27.1/ramda.min.js" integrity="sha512-rZHvUXcc1zWKsxm7rJ8lVQuIr1oOmm7cShlvpV0gWf0RvbcJN6x96al/Rp2L2BI4a4ZkT2/YfVe/8YvB2UHzQw==" crossorigin="anonymous"></script>

Similar to Ori Drori's answer, we can create generic top-down or bottom-up traversals over all possible JS types, allowing transformation at each step by passing in the current value and, where appropriate, the associated object key or array index to the provided function.

Given you want to replace the entire subtree when finding a matching node, you can process this top-down.

const mapIndexed = R.addIndex(R.map)
  
function bottomUp(f, val) {
  return R.is(Array, val)  ? mapIndexed((v, i) => f(bottomUp(f, v), i), val)
       : R.is(Object, val) ? R.mapObjIndexed((v, i) => f(bottomUp(f, v), i), val)
       : f(val, null)
}
  
function topDown(f, val) {
  function go(val, i) {
    const res = f(val, i)
    return R.is(Array, res)  ? mapIndexed(go, res)
         : R.is(Object, res) ? R.mapObjIndexed(go, res)
         : res
  }
  return go(val, null)
}

/////

const arr = [{"id":5,"name":"foo","baz":[{"a":1,"b":2},{"a":10,"b":5}],"other":{"very":{"nested":{"baz":[{"a":1,"b":2}]}}}},{"id":6,"name":"bar","baz":[]}];

const result = topDown(
  (v, k) =>  k == 'baz' ? { redacted: false } : v,
  arr
)

console.log(result)
<script src="//cdnjs.cloudflare.com/ajax/libs/ramda/0.27.0/ramda.min.js"></script>

One approach you could take is recursively finding all the paths in your object that match the provided key. An example of this can be seen in the findPaths function below. This makes use of R.chain to collect a matching path in a singleton array, or ignore unwanted keys with an empty array. If an object or array is found, it will recursively call the go function over them.

Once you have a list of all the paths you'd like to update, you can make use of R.assocPath to update the values at each path. For example, I have used reduce to iterate through each of the found paths in the redact function below.

const indexedChain = R.addIndex(R.chain)

const findPaths = (prop, obj) => {
  function go(prefix, el) {
    if (R.is(Array, el)) return indexedChain((v, i) => go(R.append(i, prefix), v), el)
    if (R.is(Object, el)) return R.chain(k => {
      if (k == prop) return [R.append(k, prefix)]
      return go(R.append(k, prefix), el[k])
    }, R.keys(el))
    return []
  } 
  return go([], obj)
}

const redact = (prop, obj) =>
  R.reduce(
    (obj_, path) => R.assocPath(path, { redacted: true }, obj_),
    obj,
    findPaths(prop, obj)
  )

//////

const arr = [
  {
    id: 5,
    name: "foo",
    baz: [{a: 1, b: 2}, {a: 10, b: 5}],
    other: {
      very: {
        nested: {
          baz: [{a: 1, b: 2}]
        }
      }
    }
  },
  {
    id: 6,
    name: "bar",
    baz: []
  }
]

//////

console.log(redact('baz', arr))
<script src="//cdnjs.cloudflare.com/ajax/libs/ramda/0.27.0/ramda.min.js"></script>

Generic Powerhouse

We could write a fairly generic nested object transformation function, which takes two callbacks, one a predicate which given a key and a value decides whether we're going to handle this node, and a second which takes a key and a value and returns a collection of key-value pairs. This lets us not only filter nodes or transform individual nodes but also create new nodes or split a single node in pieces. This has the great power / great responsibility feature, though. Writing the action callback can be uglier than we might want for simple transforms. But we then can write simpler functions built atop this to handle the simpler transforms.

Here's one implementation of such a broad function:

const transform = (pred, action) => (obj) =>
  Array .isArray (obj) 
    ? obj .map (transform (pred, action))
  : Object (obj) === obj
    ? Object .fromEntries (
        Object .entries (obj) .flatMap (
          ([k, v]) => pred (k, v) 
            ? action (k, v).map (transform (pred, action))
            : [[k, transform (pred, action) (v)]]
        )
      )
  : obj

We could use it like this:

const splitHyphensAndIncrement = transform (
  (k) => k .includes ('-'),
  (k, v) => k .split ('-') .map ((_k, i) => [_k, v + i])
)

splitHyphensAndIncrement ({a: 1, b: [{c: 2, 'x-y': 10}, {c: 3, 'x-y': 20}], 'x-y-z': 42})
//=> {a: 1, b: [{c: 2, x: 10, y: 11}, {c: 3, x: 20, y: 21}], x: 42, y: 43, z: 44}

Note what it does. It replaces single nodes including hyphens with multiple node, for instance, it replaces the x-y-x: 42 node with three nodes: x: 42, y: 43, z: 44.

This is quite powerful. But often, we don't want to deal with having to return an array of key-value pairs, when all we want to do is to convert our node into another node.

Changing Single Values

We can write a simpler transformer on top of this:

const transformSimple = (pred, action) =>
  transform (pred, (k, v) => [[k, action(v)]])

The predicate here still can use the key and the value, but the action does nothing but transform a value into another one. Here's a simple example:

const squareNumbers = transformSimple (
  (k, v) => typeof v == 'number',
  (n) => n * n
)

squareNumbers({foo: 'a', x: 3, y: 5, z: {bar: 42, x: 7}})
//=> {foo: 'a', x: 9, y: 25, z: {bar: 1764, x: 49}}

Perhaps we want a version that only squares with certain keys. We can change only the predicate:

const squareXyzNumbers = transformSimple (
  (k, v) => ['x', 'y', 'z'] .includes (k) && typeof v == 'number',
  (n) => n * n
) 

squareXyzNumbers ({foo: 'a', x: 3, y: 5, z: {bar: 42, x: 7}})
//=> {foo: "a", x: 9, y: 25, z: {bar: 42, x: 49}}

We can also use this to add sequential id numbers to each object encountered:

const addSequentialIds = transformSimple (
  (k, v) => Object (v) === v,
  ((n) => (obj) => ({_id: ++n, ...obj}))(0)
)

addSequentialIds ({foo: {bar: 'a', baz: {qux: 'b', corge: {grault: 'c'}}}})
//=> {foo: {_id :1, bar: 'a', baz: {_id: 2, qux: 'b', corge: {_id: 3, grault: 'c'}}}}

Changing Keys

We can write a simple wrapper which transforms all keys like this:

const transformKey = (fn) =>
  transform (() => true, (k, v) => [[fn(k), v]])

and use it like this:

const shoutKeys = transformKey (
  s => s.toUpperCase()
) 

shoutKeys ({foo: 1, bar: {baz: 2, qux: {corge: 3, grault: 4}}})
//=> {FOO: 1, BAR: {BAZ: 2, QUX: {CORGE: 3, GRAULT: 4}}}

Or of course we could choose to write a version that uses a predicate to decide whether to transform the key or not.

The point is that this powerful function easily offers us ways to write the simpler generic transformation functions we want, and those can lead to fairly nice code.

More Power?

Could we make this more powerful? Certainly. I'm not going to try it now, but I keep imagining changing

            ? action (k, v).map (transform (pred, action))

to

            ? action (k, v, transform (pred, action))

so that your action function receives the key, the value, and the recursive transformer, which you can choose to apply or not to the nodes you generate.

This is undoubtedly more powerful, but also gives you more chances to mess up. That might be worth the trade-off, especially if you mostly use it to create things like transformSimple or transformKey.

Original Question

Here's how we might use this to handle the original question, replacing baz: <something> with baz: {redacted: true}. We don't need the full power of transform, but can use transformSimple:

const redactBaz = transformSimple (
  (k, v) => k === 'baz',
  (v) => ({redacted: true})
)

const arr = [{id: 5, name: "foo", baz: [{a: 1, b: 2}, {a: 10, b: 5}], other: {very: {nested: {baz: [{a: 1, b: 2}]}}}}, {id: 6,name: "bar", baz: []}]

redactBaz (arr) //=>
// [
//   {id: 5, name: 'foo', baz: {redacted: true}, other: {very: {nested: {baz: {redacted :true}}}}},
//   {id: 6, name: 'bar', baz: {redacted: true}}
// ]

In Action

We can see all this in action in this snippet:

const transform = (pred, action) => (obj) =>
  Array .isArray (obj) 
    ? obj .map (transform (pred, action))
  : Object (obj) === obj
    ? Object .fromEntries (
        Object .entries (obj) .flatMap (
          ([k, v]) => pred (k, v) 
            ? action (k, v) .map (transform (pred, action))
            : [[k, transform (pred, action) (v)]]
        )
      )
  : obj

const res1 = transform (
  (k) => k .includes ('-'),
  (k, v) => k .split ('-') .map ((_k, i) => [_k, v + i])
) ({a: 1, b: [{c: 2, 'x-y': 10}, {c: 3, 'x-y': 20}], 'x-y-z': 42})

const transformSimple = (pred, action) =>
  transform (pred, (k, v) => [[k, action(v)]])

const res2 = transformSimple (
  (k, v) => typeof v == 'number',
  (n) => n * n
)({foo: 'a', x: 3, y: 5, z: {bar: 42, x: 7}})

const res3 = transformSimple (
  (k, v) => ['x', 'y', 'z'] .includes (k) && typeof v == 'number',
  (n) => n * n
) ({foo: 'a', x: 3, y: 5, z: {bar: 42, x: 7}})

const res4 = transformSimple (
  (k, v) => Object (v) === v,
  ((n) => (obj) => ({_id: ++n, ...obj}))(0)
)({foo: {bar: 'a', baz: {qux: 'b', corge: {grault: 'c'}}}})


const transformKey = (fn) =>
  transform (() => true, (k, v) => [[fn(k), v]])

const res5 = transformKey (
  s => s.toUpperCase()
) ({foo: 1, bar: {baz: 2, qux: {corge: 3, grault: 4}}})

const arr = [{id: 5, name: "foo", baz: [{a: 1, b: 2}, {a: 10, b: 5}], other: {very: {nested: {baz: [{a: 1, b: 2}]}}}}, {id: 6,name: "bar", baz: []}]

const res6 = transformSimple (
  (k, v) => k === 'baz',
  (v) => ({redacted: true})
) (arr)


console .log (res1)
console .log (res2)
console .log (res3)
console .log (res4)
console .log (res5)
console .log (res6)
.as-console-wrapper {max-height: 100% !important; top: 0}

Ramda

I'm one of the founders of Ramda and a big fan, but I rarely use it in recursive situations. It isn't particularly designed to help with them.

But if I was writing this in an environment that was already using Ramda, it can certainly help around the edges. is, map, pipe, toPairs, fromPairs, chain are all slightly nicer than their native counterparts, and there's a reasonable ifElse lurking in the Object branch. So we could write this like:

const transform = (pred, action) => (obj) =>
  is (Array) (obj) 
    ? map (transform (pred, action)) (obj)
  : is (Object) (obj)
    ? pipe (
        toPairs,
        chain (apply (ifElse (
          pred, 
          pipe (action, map (transform(pred, action))), 
          (k, v) => [[k, transform (pred, action) (v)]]
        ))),
        fromPairs
      ) (obj)
  : obj

There also looks to be a cond lurking at the root of this function (note that all the conditions and consequents end by applying our parameter, except the final clause, which could easily be replaced by identity), but I'm not quite sure how to make that work with the recursion.

It's not that Ramda offers nothing here. I slightly prefer this version to the original. But there's not enough difference that I would introduce Ramda to a codebase that wasn't using it just for those minor advantages. And if I were to get that cond working, it might well be a clearly better version.

Update

I did figure out how to get that cond working, and it's nicer code, but still not quite what I'd like. That version looks like this:

const transform = (pred, action) => (o) => cond ([
  [is (Array), map (transform (pred, action))],
  [is (Object), pipe (
    toPairs,
    chain (apply (ifElse (pred, pipe (action, map (transform (pred, action))), (k, v) => [[k, transform (pred, action) (v)]]))),
    fromPairs
  )],
  [T, identity]
]) (o)

If I didn't have to have the (o) => cond ([...]) (o) and could just use cond ([...]) I would be very happy with this. But the recursive calls inside the cond won't permit that.

Related