gRPC unary-streaming continues after disconnect / blocks on interrupt

Viewed 1509

I have a simple unary-stream method Subscribe which send the most recent item seen by the connected client on a queue. If no new item has been pushed in a second, the most recently seen item is sent again for a heartbeat.

class Service(proto_grpc.ServiceServicer):
    def __init__(self, queue):
        self.queue = queue

    def Subscribe(self, request, context):
        index = 0
        state = self.queue.get(index, timeout=None)

        while state:
            temp = self.queue.get(index, timeout=1)
            if temp:
                state = temp
                index = index + 1

            yield state

The remaining relevant portion of the program is given below.

def main(server, queue):
    server.start()

    try:
        while True:
            queue.append('foo')
        time.sleep(1)
    finally:
        server.stop(None)

Logging shows that if a client subscribes and then disconnects (ungracefully), the Subscribe method will continue to process. Each disconnection also seems to be unavailable to new clients.

Also strangely, if the server process is interrupted, then the server is stopped, the client is disconnected, and the server blocks on the yield statement.

What is the proper way to deal with ungraceful clients being removed from a streaming RPC?

EDIT: A simplified version of the servicer that displays the same behavior.

class Service(proto_grpc.ServiceServicer):
    def Subscribe(self, request, context):
        print 'subscribed'

        while context.is_active():
            time.sleep(1)
            print 'sending'
            yield some_proto_payload
            print 'sent'

        print 'finished'

Once connected, 'sending/sent' pairs will be printed regardless of the connection status of the client. When the server stops after a disconnect, output hangs between the last 'sending' and 'sent' messages.

0 Answers
Related