Sorting items into an array by property

user_6238395 0
Viewed 139

Consider a list containing a number of items, each containing information about the same N properties. Each of these properties can take on a certain limited (but perhaps unknown) number of discrete values. The list is not necessarily ordered.

I would like to sort these items into an N-dimensional array such that each property varies only along its own axis, i.e. is constant within any slice along that axis.

Simplified Example:

# Three properties
prop_1 = ['01','02','03','07']
prop_2 = ['foo','bar','baz']
prop_3 = ['yellow','red']

from itertools import product

# Consider this as the input
string_list = ['_'.join(s) for s in product(prop_1, prop_2, prop_3)]

# HOWEVER...

from random import shuffle

# Inputs may be unsorted
shuffle(string_list)

I now want to organize string_list into an array of shape (4,3,2) such that the first property varies along the first axis, & so on. That is, the expected output would be:

array([[['01_foo_yellow','01_foo_red'],
        ['01_bar_yellow','01_bar_red'],
        ['01_baz_yellow','01_baz_red']],

       [['02_foo_yellow','02_foo_red'],
        ['02_bar_yellow','02_bar_red'],
        ['02_baz_yellow','02_baz_red']],

       [['03_foo_yellow','03_foo_red'],
        ['03_bar_yellow','03_bar_red'],
        ['03_baz_yellow','03_baz_red']],

       [['07_foo_yellow','07_foo_red'],
        ['07_bar_yellow','07_bar_red'],
        ['07_baz_yellow','07_baz_red']]])

Each property is then constant within any slice along its own axis, i.e.:

A[3,...] # All strings containing '07' as property 1
A[:,1,:] # All strings containing 'bar' as property 2
A[...,0] # All strings containing 'yellow' as property 3

The method should be robust even when there are missing items. For example, if we remove '02_bar_yellow' and '03_baz_red' from the inputs, the shape of the output array should remain unchanged, with None where these entries would otherwise have been sorted:

array([[['01_foo_yellow','01_foo_red'],
        ['01_bar_yellow','01_bar_red'],
        ['01_baz_yellow','01_baz_red']],

       [['02_foo_yellow','02_foo_red'],
        [ None,          '02_bar_red'],
        ['02_baz_yellow','02_baz_red']],

       [['03_foo_yellow','03_foo_red'],
        ['03_bar_yellow','03_bar_red'],
        ['03_baz_yellow', None.      ]],

       [['07_foo_yellow','07_foo_red'],
        ['07_bar_yellow','07_bar_red'],
        ['07_baz_yellow','07_baz_red']]])

The Problem:

While the above illustrates the general idea, I am actually trying to get this to work for a set of regex match objects whose 'properties' are its capturing groups.

import re

pattern = '(\d+)_(\w+)_(\w+)'
regex = re.compile(pattern)

# Consider this as the input
matches = [re.match(s) for s in string_list]

I then want to sort according to the values given by the group() method of each match object.

Although it does not quite approach a solution, I can get the items sorted by slice along an axis using itertools.groupby():

# Sort by the first capturing group
groupings = itertools.groupby(matches, key=lambda m: m.groups()[0])

grouped_strings = [[m.string for m in g] for n,g in groupings]

So the contents of grouped_strings[3] are the same as the slice A[3,...] given in the first example. However, these entries are given as a flattened array.

It occurs to me that I should be able to use itertools.groupby iteratively to achieve the proper sorting, but I can't quite figure it out. At the same time, I wonder if there is a simpler or more 'pythonic' way to achieve this.

grouping sorting arrays python regex
2 Answers

If you don't know the length of the axes initially, you need to get them, e.g., by using set or np.unique, which will also sort the result and compute an inverse index for you.

You can then make a boolean mask indicating which items were found and which weren't. It is trivial to map mask indices to the sorted label arrays.

Let's say you have three matches of interest, as in the example:

# Findall is the lazy shortcut that assumes only one match
matches = np.array([regex.findall(s) for s in string_list])
p1, i1 = np.unique(matches[:, 0], return_inverse=True)
p2, i2 = np.unique(matches[:, 1], return_inverse=True)
p3, i3 = np.unique(matches[:, 2], return_inverse=True)
mask = np.zeros((p1.size, p2.size, p3.size), dtype=bool)
mask[i1, i2, i3] = True

p* are the labels for the axes of your array. Since np.unique is sort-based, these will be unique and sorted.

i* are the indices that map elements of string_list to each axis. This means that the fancy index i1, i2, i3 indicates the locations in the boolean matrix corresponding to a match that was found.

This can be easily generalized with a loop over an arbitrary number of capturing groups.

# note the transpose
matches = np.array([regex.findall(s) for s in string_list]).T
# must be list because ragged
axes = [np.unique(x, return_inverse=True) for x in matches]
mask = np.zeros(tuple(x[0].size for x in axes), dtype=bool)
mask[tuple(x[1] for x in axes)] = True
user_29887300

You could try with np.reshape with the intial string:

import numpy as np
output=np.array(string_list).reshape(4,3,2) 

print(output)

Output:

[[['01_foo_yellow' '01_foo_red']
  ['01_bar_yellow' '01_bar_red']
  ['01_baz_yellow' '01_baz_red']]

 [['02_foo_yellow' '02_foo_red']
  ['02_bar_yellow' '02_bar_red']
  ['02_baz_yellow' '02_baz_red']]

 [['03_foo_yellow' '03_foo_red']
  ['03_bar_yellow' '03_bar_red']
  ['03_baz_yellow' '03_baz_red']]

 [['07_foo_yellow' '07_foo_red']
  ['07_bar_yellow' '07_bar_red']
  ['07_baz_yellow' '07_baz_red']]]
user_136762020
Related