datetime timestamp using Python with microsecond level accuracy

user_11163122 0
Viewed 522

I am trying to get timestamps that are accurate down to the microsecond on Windows OS and macOS in Python 3.10+.

On Windows OS, I have noticed Python's built-in time.time() (paired with datetime.fromtimestamp()) and datetime.datetime.now() seem to have a slower clock. They don't have enough resolution to differentiate microsecond-level events. The good news is time functions like time.perf_counter() and time.time_ns() do seem to use a clock that is fast enough to measure microsecond-level events.

Sadly, I can't figure out how to get them into datetime objects. How can I get the output of time.perf_counter() or PEP 564's nanosecond resolution time functions into a datetime object?

Note: I don't need nanosecond-level stuff, so it's okay to throw away out precision below 1-μs).

Current Solution

This is my current (hacky) solution, which actually works fine, but I am wondering if there's a cleaner way:

import time
from datetime import datetime, timedelta
from typing import Final

IMPORT_TIMESTAMP: Final[datetime] = datetime.now()
INITIAL_PERF_COUNTER: Final[float] = time.perf_counter()


def get_timestamp() -> datetime:
    """Get a high resolution timestamp with μs-level precision."""
    dt_sec = time.perf_counter() - INITIAL_PERF_COUNTER
    return IMPORT_TIMESTAMP + timedelta(seconds=dt_sec)
timestamp time datetime python
1 Answers

That's almost as good as it gets, since the C module, if available, overrides all classes defined in the pure Python implementation of the datetime module with the fast C implementation, and there are no hooks.
Reference: python/cpython@cf86e36

Note that:

  1. There's an intrinsic sub-microsecond error in the accuracy equal to the time it takes between obtaining the system time in datetime.now() and obtaining the performance counter time.
  2. There's a sub-microsecond performance cost to add a datetime and a timedelta.

Depending on your specific use case if calling multiple times, that may or may not matter.

A slight improvement would be:

INITIAL_TIMESTAMP: Final[float] = time.time()
INITIAL_TIMESTAMP_PERF_COUNTER: Final[float] = time.perf_counter()

def get_timestamp_float() -> float:
    dt_sec = time.perf_counter() - INITIAL_TIMESTAMP_PERF_COUNTER
    return INITIAL_TIMESTAMP + dt_sec

def get_timestamp_now() -> datetime:
    dt_sec = time.perf_counter() - INITIAL_TIMESTAMP_PERF_COUNTER
    return datetime.fromtimestamp(INITIAL_TIMESTAMP + dt_sec)

Anecdotal numbers

Windows:

# Intrinsic error
timeit.timeit('datetime.now()', setup='from datetime import datetime')/1000000  # 0.31 μs
timeit.timeit('time.time()', setup='import time')/1000000                       # 0.07 μs

# Performance cost
setup = 'from datetime import datetime, timedelta; import time'
timeit.timeit('datetime.now() + timedelta(1.000001)', setup=setup)/1000000            # 0.79 μs
timeit.timeit('datetime.fromtimestamp(time.time() + 1.000001)', setup=setup)/1000000  # 0.44 μs

# Resolution
min get_timestamp_float() delta: 239 ns

Windows and macOS:

Windows macOS
# Intrinsic error
timeit.timeit('datetime.now()', setup='from datetime import datetime')/1000000 0.31 μs 0.61 μs
timeit.timeit('time.time()', setup='import time')/1000000 0.07 μs 0.08 μs
# Performance cost
setup = 'from datetime import datetime, timedelta; import time' - -
timeit.timeit('datetime.now() + timedelta(1.000001)', setup=setup)/1000000 0.79 μs 1.26 μs
timeit.timeit('datetime.fromtimestamp(time.time() + 1.000001)', setup=setup)/1000000 0.44 μs 0.69 μs
# Resolution
min time() delta (benchmark) x ms 716 ns
min get_timestamp_float() delta 239 ns 239 ns

239 ns is the smallest difference that float allows at the magnitude of Unix time, as noted by Kelly Bundy in the comments.

x = time.time()
print((math.nextafter(x, 2*x) - x) * 1e9)  # 238.4185791015625

Script

Resolution script, based on https://www.python.org/dev/peps/pep-0564/#script:

import math
import time
from typing import Final

LOOPS = 10 ** 6

INITIAL_TIMESTAMP: Final[float] = time.time()
INITIAL_TIMESTAMP_PERF_COUNTER: Final[float] = time.perf_counter()

def get_timestamp_float() -> float:
    dt_sec = time.perf_counter() - INITIAL_TIMESTAMP_PERF_COUNTER
    return INITIAL_TIMESTAMP + dt_sec

min_dt = [abs(time.time() - time.time())
          for _ in range(LOOPS)]
min_dt = min(filter(bool, min_dt))
print("min time() delta: %s ns" % math.ceil(min_dt * 1e9))

min_dt = [abs(get_timestamp_float() - get_timestamp_float())
          for _ in range(LOOPS)]
min_dt = min(filter(bool, min_dt))
print("min get_timestamp_float() delta: %s ns" % math.ceil(min_dt * 1e9))
user_86017600
Related