How to establish a happens-before relationship between a request handling thread and a SocketChannel selector thread?

user_5191913 0
Viewed 213

Consider a request-response protocol.

We spawn a thread to perform a select() loop for reads and writes on an accepted non-blocking SocketChannel. That might look something like

while (!isStopped()) {
    selector.select();
    Iterator<SelectionKey> selectedKeys = selector.selectedKeys().iterator();

    while (selectedKeys.hasNext()) {
        SelectionKey selectedKey = selectedKeys.next();
        selectedKeys.remove();
        Context context = (Context) selectedKey.attachment();
        if (selectedKey.isReadable()) {
            context.readRequest();
        } else /* if (selectedKey.isWritable()) */ {
            context.writeResponse();
        }
    }
}

where Context is just a container for the corresponding SocketChannel, a buffer and logic to read into it and write from it. The readRequest might look like

public void readRequest() {
    // read all content
    socketChannel.read(requestBuffer);
    // not interested anymore
    selectionKey.interestOps(0);
    executorService.submit(() -> {
        // handle request with request buffer and prepare response
        responseBuffer.put(/* some response content */); // or set fields of some bean that will be serialized

        // notify selector, ready to write
        selectionKey.interestOps(SelectionKey.OP_WRITE);
        selectionKey.selector().wakeup(); // worried about this
    });
}

In other words, we read from the socket channel, populate some buffer and hand off the handling to some other thread. That thread does the handling and prepares a response which it stores in a response buffer. It then notifies the selector that it wants to write and wakes it up.

The Javadoc for Selector#wakeup() doesn't mention any happens-before relationship so I'm worried the selector thread might see the response buffer (or some intermediate object) in an inconsistent state.

Is that a possible scenario? If it is, what's the correct way to hand off a response to be written to a SocketChannel by a Selector loop thread? (Publishing the response through some volatile field? Using a SelectionKey attachment? Some other form of synchronization?)

happens-before nio sockets multithreading java
2 Answers

Firstly, you don't need to notify the selector about wanting to write. You just write. Only in the case where the write returned zero does the selector or its thread need to be involved.

Secondly, the happens-before relationship occurs as a result of the selector's three levels of synchronization, provided you also do a synchronization, as below.

Your code may block in the interestOps() call if the selector is currently selecting. The possibility is not excluded by the Javadoc. You need to do the operations in the right order:

  1. Wakeup.
  2. Synchronize on the selector.
  3. Call interestOps().

The combination of (2) and the selector's own internal synchronizations establishes any necessary happens-before relationships.

user_2074210

The documentation on Selector says the following:

The selection operations synchronize on the selector itself, on the key set, and on the selected-key set, in that order.

The happens-before relation is defined in the Java Language Specification, Chapter 17, as including the synchronizes-with relation.

Even so, you should synchronize properly on the attached object. That's your object, it's your duty. Assuming only your code writes to responseBuffer in the executor's thread, and only the selector thread reads from it after you say you're interested on write availability, you have enough synchronization.

What may surprise you is that you're getting that synchronization from interestOps(...), even before wakeup().

From my experience, if you have a hard time trying to achieve proper synchronization through library utilities (in this case, a selector), you'd better synchronize your object yourself, e.g. with the synchronize statement on the object itself, a ReentrantLock, some other common synchronization object you use on your object's operations, etc. You lose a tiny bit of performance (actually, insignificant in most cases, assuming you don't block within the guarded section) to keep calm.

user_8005240
Related