Cython program (print Hello world) much slower than pure Python

Question

I am new to Cython. I've written a super simple test programm to access benefits of Cython. Yet, my pure python is alot faster. Am I doing something wrong? test.py:

import timeit
imp = '''
import pyximport; pyximport.install()
from hello_cy import hello_c
from hello_py import hello_p
'''

code_py = '''
hello_p()
'''

code_cy = '''
hello_c()
'''

print(timeit.timeit(stmt=code_py, setup=imp))
print(timeit.timeit(stmt=code_cy, setup=imp))

hello_py.py:

def hello_p():
    print('Hello World')

hello_cy.pyx:

from libc.stdio cimport printf

cpdef void hello_c():
    cdef char * hello_world = 'hello from C world'
    printf(hello_world)

hello_py timeit takes 14.697s

hello_cy timeit takes 98s

Am I missing something? How can I make my calls to cpdef functions run faster?

Thank you very much!

user_3545273 · Answer

I strongly suspect a problem in your configuration.

I have (partially) reproduced your tests in Windows 10, Python 3.10.0, Cython 0.29.26, MSVC 2022, and got quite different results

Because in my tests the Cython code is slightly faster. I made 2 changes:

in hello_cy.pyx, to make both code closer, I have added the newline:
```
  ...
  printf("%s
", hello_world)
```

In the main script I have splitted the call of the functions and the display of the times:

  ...
  pyp = timeit.timeit(stmt=code_py, setup=imp)
  pyc = timeit.timeit(stmt=code_cy, setup=imp)

  print(pyp)
  print(pyc)

When I run the script I get (after pages of hello...):

...
hello from C world
hello from C world
19.135732599999756
14.712803700007498

Which looks more like what could be expected...

Anyway, we do not really know what is tested here, because as much as possible, the IO should not be tested because it depends of a lot of things outside of the programs themselves.

user_4657412 · Answer

It just isn't a meaningful test - the two functions aren't the same:

Python print prints a string followed by a newline. I think it then flushes the output.
printf scans the string for formatting characters (e.g. %s) and then prints it without an extra newline. By default printf is line bufferred (i.e. flushes after each new line). Since you never post a new line or flush it, then it may be slowed down by managing an increasingly huge buffer.

In summary, don't be mislead by fairly meaningless microbenchmarks. Especially for terminal IO which is rarely actually a limiting factor.

user_17855920 · Answer

It takes too long because pyximport compiles the cython code on the fly (so you are measuring also compilation from cython to C and compilation C code to native library). You should measure calling already compiled code - see https://cython.readthedocs.io/en/latest/src/quickstart/build.html#building-a-cython-module-using-setuptools

Cython program (print Hello world) much slower than pure Python

3 Answers