From google docs:
Because the BigQuery client uses the third-party requests library by default and the BigQuery-Storage client uses grpcio library, both are safe to share instances across threads. In multiprocessing scenarios, the best practice is to create client instances after
multiprocessing.Poolormultiprocessing.Processinvokesos.fork().
On the one hand "safe to share instances across threads" is mentioned, on the other, to create client instances after "os.fork()" is done.
And I don't understand if I can create one client and pass to threads to use, or if each thread has to initialize client itself separately.
I realize multithreading with the following code:
with multiprocessing.Pool(processes=NUMBER_OF_PROCESSES) as proc:
proc.map(my_bq_func, arguments_list)