Why is the creation of Python protobuf messages so slow?

Viewed 1856

Say I have a message defined in test.proto as:

message TestMessage {
    int64 id = 1;
    string title = 2;
    string subtitle = 3;
    string description = 4;
}

And I use protoc to convert it to Python like so:

protoc --python_out=. test.proto

timeit for PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python:

from test_pb2 import TestMessage

%%timeit
tm = TestMessage()
tm.id = 1
tm.title = 'test title'
tm.subtitle = 'test subtitle'
tm.description = 'this is a test description'

6.75 µs ± 152 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)

timeit for PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=cpp:

1.6 µs ± 115 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)

Compare that to just a dict:

%%timeit
tm = dict(
    id=1,
    title='test title',
    subtitle='test subtitle',
    description='this is a test description'
)

308 ns ± 2.47 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)

This is also only for one message. Protobuf cpp implementation is about 10.6µs for my full project.

Is there a way to make this faster? Perhaps compiling the output (test_pb2)?

0 Answers
Related