I realized that outputs of a continuous function (takes a vector, returns a scalar) that I wrote in python are discretized at the resolution of 9.536743e-7.
I googled this number and learned that some websites say that 1bit=9.536743e-7 megabit. Isn't 1 bit = 10e-8 megabit? Why am I seeing this number everywhere and why do I see it in my code?
My code was written in jax.numpy.
When discussing memory, "mega" typically means 2^20 = 1048576. And, yes: 1/1048576 = 9.536743e-7
Whereas, when discussing bandwidth/speed, "mega" typically means 1000000. However, when discussing memory, bytes are usually used (vs bits for bandwidth/speed); so that part is strange. So I suppose you could say the time for 1 bit on a 1 megabit link is 9.536743e-7 seconds. But it is atypical at best. I don't see how this 1/x thinking would apply to memory.
Note that this number cannot be represented exactly as a float; here is the closest 64 bit float value:
>>> print "%1000.1000f\n" % 0.000000953674316
0.0000009536743159999999753652735225151193532155957655049860477447509765625000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
>>>