Group array elements based on value and get median of other value

Question

I'm trying to create a visualization based on the data from the Spotify audio analysis. Here is a part of the data:

{
  ...
  "segments": [
    {
      "start": 0,
      "duration": 0.20934,
      "confidence": 1,
      "loudness_start": -60,
      "loudness_max_time": 0.06898,
      "loudness_max": -22.328,
      "loudness_end": 0,
      ...
    },
    {
      "start": 0.20934,
      "duration": 0.10907,
      "confidence": 0.772,
      "loudness_start": -35.399,
      "loudness_max_time": 0.01327,
      "loudness_max": -25.11,
      "loudness_end": 0,
      ...
    },
    ...
  ],
  ...
}

What I want to do:

Group these segments based on the start value. Example: All segments from second 0 to second 1 are a group, all segments from second 1 to second 2 and so on...
Every group should be a new object with all the values combined (e.g. the median of all loudness_max values, the median of all loudness_start values, etc...)

I already tried using d3s group and rollup functions but I couldn't get it to work so far. How would one achieve this using js/d3?

I hope this description is comprehensible, thanks in advance for your help!

user_14920188 · Answer

One approach is to use d3.bin() to create buckets, and map the buckets to aggregate the properties with an aggregation function, such as d3.median().

The d3.bin() method creates groups of data based on a given bucketing definition (d3 calls it thresholds). The output of d3.bin() is the same data you input, but with arrays around the buckets. For instance, an array [0.2, 0.8, 1.2, 1.8] binned in the thresholds [0, 1, 2] becomes [ [0.2, 0.8], [1.2, 1.8] ].

You can use the d3.bin().value() method to specify an accessor, which in your case is a function that accesses the start property d => d.start.

Since you want the thresholds to be whole seconds, there are a few pre-processing calculations to compute the thresholds; I'll explain them in the comments of the following code snippet:

    // Compute the max value of the data and round up to get the whole second.
    const maxValue = Math.ceil(d3.max(data, d=> d.start))
    
    // Define the bin thresholds (In your case, an array of whole seconds: 0, 1, ... , maxValue)
    const thresholds = d3.range(maxValue+1)

    // Define the min and max values of the buckets
    const domain = [0, maxValue]
    
    // Create the binner function 
    const binner = d3.bin().value(d=>d.start).thresholds(thresholds).domain(domain)

    // Bin the data
    const binned = binner(data)
    
    // Map each bin array and return
    const medians = binned.map(bin => {
        return {
          medianDuration: d3.median(bin, b=>b.duration),
          medianConfidence: d3.median(bin, b=>b.confidence),
            // ... The other properties repeat the same pattern

            /* You can also extract three additional values: the amount of data 
            in the bin, the start of the bin, and the end of the bin; */
          dataPoints: bin.length,
          bucketMin: bin.x0,
          bucketMax: bin.x1
      }
    })

Working example:

const data = getData()
const maxValue = Math.ceil(d3.max(data, d=> d.start))

const thresholds = d3.range(maxValue+1)
const domain = [0, maxValue]

const binner = d3.bin().value(d=>d.start).thresholds(thresholds).domain(domain)
const binned = binner(data)

const medians = binned.map(bin => {
    return {
    medianDuration: d3.median(bin, b=>b.duration),
    dataPoints: bin.length,
    bucketMin: bin.x0,
    bucketMax: bin.x1

  }
})

console.log(medians)

function getData() {
  return [{
      "start": 0.2,
      "duration": 0.20934,
}, {
      "start": 0.8,
      "duration": 0.30934,
},
{
      "start": 1.2,
      "duration": 0.20934,
},
{
      "start": 1.8,
      "duration": 0.10934,
},
{
      "start": 2.2,
      "duration": 0.30934,
},
{
      "start": 2.8,
      "duration": 0.40934,
}]
}

Group array elements based on value and get median of other value

1 Answers