Numba v0.44: cannot reflect element of reflected container

user_5623899 0
Viewed 1782

I am new to numba and am struggling at every turn to get what I think is simple to work in nopython mode.

For example, inspired by this question coalescing-ranges I have written the following function:

@njit
# @jit
def coalesce(ranges):
    coalesced = []
    for i, (label_a, start_a, stop_a) in enumerate(ranges):
        append_flag = True
        for j, (label_b, start_b, stop_b) in enumerate(coalesced):
            if label_a != label_b: # not of same type
                continue
            elif stop_a < start_b: # a does not start and then overlap b
                continue
            elif stop_b < start_a: # b does not start and then overlap a
                continue
            else:
                # same type and overlap, merge into i, do not append
                append_flag = False
                coalesced[j] = [label_a, min([start_a, start_b]), max([stop_a, stop_b])]
                break
        if append_flag:
            coalesced.append([label_a, start_a, stop_a])
    return coalesced

It assumes that it is passed in a list of lists. Each sub list consists of only integers [type, start, stop] and this function is mention to merge similar typed ranges which overlap e.g.

[
    [1, 10, 100],
    [0, 50, 75],
    [1, 50, 150],
    [0, 10, 100],    
    [0, 200, 300],    
    [0, 15, 150]
]
# becomes
[
    [0, 10, 150],
    [0, 200, 300],
    [1, 10, 150]
]

This function works with @jit (although it spits out a ton of warnings). When calling this function with @njit and the above list:

TypeError: Failed in nopython mode pipeline (step: nopython mode backend)
cannot reflect element of reflected container: reflected list(reflected list(int64))

I have no idea what this means or why this fails.

numba python-3.x python
2 Answers

Numba now offers Typed Lists, these accept a list of lists. As far as I can see they still need to be appended individually though (because the TypedList.extend method doesn't accept LofL). This leads to:

from numba.typed import List  # As per the docs, since it's in beta, it needs to be imported explicitly

lol = [[1, 10, 100], [0, 50, 75], [1, 50, 150], [0, 10, 100], [0, 200, 300], [0, 15, 150]]

nb_list = List()
for lst in lol:
    nb_list.append(lst)

coalesce(nb_list)
user_23854900

Numba as of 0.44 does not support list of lists as inputs to functions in nopython mode. See:

http://numba.pydata.org/numba-doc/latest/reference/pysupported.html#list-reflection

The warnings you are seeing when using jit are related to the fact that falling back to object mode, but in future releases of numba, this will raise an error rather than falling back. See:

http://numba.pydata.org/numba-doc/latest/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit

If you can convert your input to a 2D numpy array, then you can get around this limitation.

user_3929490
Related