Performance while joining strings from mapping

Question

I am looking at performance while joining strings from a mapping (Mapping[str, str]).

I have these two simple functions:

def nested(sequence, mapping):
    
    string = ""
    
    for x in sequence:
        
        string += mapping[x]
    
    return string

and

def flat(sequence, mapping):
    
    return "".join(mapping[x] for x in sequence)

In my project, I found a clear loss in performance while using the "flat" solution. Initially, I choose that solution because: "Flat is better than nested.", but quickly moved to the "nested" solution.

I ran a timeit test with simpler sequences and mappings:

letters = (
    "a","b","c","d","e","f","g","h","i","j","k","l","m",
    "n","o","p","q","r","s","t","u","v","w","x","y","z"
)

mymapping = {x: x for x in letters}

mysequence = "mysequence"

print(timeit.timeit("nested(mysequence, mymapping)", globals=globals()))

print(timeit.timeit("flat(mysequence, mymapping)", globals=globals()))

and obtained:

In [683]: 0.675819274969399
0.965234256349504

This is 43% faster for the "nested" solution. I observed several times faster situations in my project.

In the docs, we can read: "The preferred, fast way to concatenate a sequence of strings is by calling ''.join(sequence)". But I clearly see that "+" operator is faster in this configuration.

While I have no problem to stay with the "nested" solution:

Am I missing another "flat" solution? (no sum() possible with strings)

Do you have an explanation for this behavior?

Thanks.

user_4934061 · Answer

When you use list comprehension, join is faster ony my end:

def flat_list(sequence, mapping):
    return "".join([mapping[x] for x in sequence])

2.5292927          # nested
2.3440613999999997 # flat_list
6.530926599999999  # flat

My guess is that it has something to do with this. There seems to be some small overhead with the generator expression and repeated 10000 times this differences becomes visible.

user_2403000 · Answer

Am I missing another "flat" solution? (no sum() possible with strings)

The answer by Josef looks good so I'm just responding to this part. There's a much more elegant way (in my opinion), which seems to be faster than the alternatives too.

def remap(sequence, mapping):
    return ''.join(map(mapping.get, sequence))

# mysequence
1.098659     # nested
1.1296278    # flat_list
1.4904282    # flat
0.9128849    # remap

# mysequence * 10
8.5377648    # nested
8.814077     # flat_list
6.1268931    # flat
5.555387     # remap

Performance while joining strings from mapping

2 Answers