TLDR;
I need to run the gRPC Cpp client library as a single thread. From what I can tell, initializing grpc creates two threads for Executors (default executor and resolver executor) and one to two threads for Timers (from timer_manager). I can turn these threads off after creation but I cant figure out how to prevent them from being created. Is there a way to stop their creation using any of the APIs?
Explanation
Threading in Completion Queue, Executors, and Execution Contexts
Lets say we have a cpp file with a completion queue:
using grpc::CompletionQueue;
CompletionQueue* globalCompletionQueuePtr;
void main()
{
globalCompletionQueuePtr = new CompletionQueue;
}
Having done this we then have this sequence kick off:
- Creating a CompletionQueue in this way initializes grpc (
grpc_init()ininit.cc) grpc_initwill then callgrpc_iomgr_initwhich then calls InitAll off ofgrpc_core::Executor- In
executor.cc,InitAllcreates the default and resolver executors and then callsInit()on each. Initthen callsSetThreading(true)which goes about starting up an execution thread for each executor.
Now we have two threads spun up separate from the main thread, one for the default executor and one for resolver executor. Not looking any farther into this, I can then remove the threads by calling grpc_core::Executor::SetThreadingAll(false); after creating the completion queue but this means that the threads will create and start work and then be terminated.
Questions about executors, the completion queue, and execution contexts:
- How do the executors relate to the poll engine? I see that the executors run closures but are they responsible for executing all closures? I can run closures when they are turned off so I must assume thats happening on the main thread. Is that right?
- Calling
AsyncNexton the completion queue above drives the operations on the queue to finish as the documentation says. I can push operations onto the queue (withgrpc_cq_begin_opandgrpc_cq_end_op) and I can grab the underlying pollset, create a pollent, and use that to schedule calls myself. In this way, it looks like the queue tracks the state of operations but is not itself responsible for the operations doing work. Is that right? - I know that certain calls into grpc need the
grpc_core::ExecCtx exec_ctx;context object created on the stack. How does the stack ctx interact with the resolver and default executors? Does it? - Is it possible to init grpc without the executors? Calling
SetThreading(false)seems to keep the library working but I dont want to create threads and then kill them.
Threading in Timers
Separate from the completion queue, after the iomgr init:
grpc_initininit.cclater callsgrpc_iomgr_startiniomgr.ccwhich callsgrpc_timer_manager_initintimer_manager.cc- The last thing
grpc_timer_manager_initdoes is callstart_threads() start_threads()checksg_threadedto see it needs to start some threads and then does so by callingstart_timer_thread_and_unlock
Now theres a timer thread which will figure out how long until the next timer fires, sleep until that time, then wake up and fire the timers. If we run out of threads, we will start up another thread as long as we are in threaded mode (g_threaded). The code basically puts us in threaded mode no matter what, but there is a call grpc_timer_manager_set_threading(false); from timer_manager that will stop all the timer threads.
Questions about timers:
- For these timer threads, what is their main use relative to grpc calls? Are the timers mostly an internal construct or are they used by the public API in some way? Are they responsible for enforcing the deadlines on closures?
- Is there a way to init grpc without the timer threads? I can turn them off as stated above but its got the same problem as the executors in that they get created and then I destroy them afterwards.
- Will turning off the timer threads have any negative impact on the operations of gRPC such as deadlines no longer working? Will gRPC spin up new threads even after calling
grpc_timer_manager_set_threading? Are the timers resolved on the main thread in a coroutine way similiar to the closures by calling AsyncNext on the queue without threads? Is it already doing that?
Extra Context
Finally, is there anything else in the library that will spin up threads that I'm blind to seeing?
The reason I ask all these questions is that I need to run grpc inside an application where the application provides a single thread for the library to run on. Performance degradation from the lack of threads is not a concern.
If anything I have said here is inaccurate, please do correct me. I know I am working with an imperfect understanding of the grpc cpp library.
Thanks in advance for any answers and anyone who takes the time to read through this and provide support. I greatly appreciate it!
UPDATE: Why do I need a single thread?
I have a specific hardware environment where the actual application that will run the gRPC client will be managing several threads. Each thread will get a time slice to run in and must be done at the end of that time slice. While extra threads can spin up during that thread's time slice, they must all be finished when the time slice is over so that the next thread, when given its time slice, has all the hardware resources available to it.