Does HTTP allow the server to start output before consuming all input? If so, how to access such a server in Java?

user_4858665 0
Viewed 110

Note: this question is closely related to Is it acceptable for a server to send a HTTP response before the entire request has been received? with the difference that (1) I'm not sending an error, I'm sending a 200 OK, and (2) I control both the client and server, so don't really care about browser support.

Context: I am implementing a Java HTTP client and server for managing files. In particular an "upload" query contains a file path and the file body, and the server responds with a numerical identifier for the file. However if a file with the same path has already been uploaded, the server will simply respond with the previously generated identifier.

Concretely: if I write the server as follows (sparkjava)

put(url, (req, res) -> {
  Item existing = lookForExistingItem(req);
  if (existing != null) {
     return existing.getId();
  }
  /* Otherwise, consume input, save, generate id and return that */
});

... then the server will respond with the id and close the connection before the client finished sending data. If I write the client as follows:

final HttpURLConnection connection = (HttpURLConnection) new URL(...).openConnection();
connection.setDoOutput(true);
connection.setRequestMethod("PUT");
ByteStreams.copy(fileInput, connection.getOutputStream());
final String response = CharStreams.toString(new InputStreamReader(connection.getInputStream()));

then an IOException is thrown during the copy operation due to the closed connection. After that point I am not able to access the connection's InputStream anymore.

My Question: how can I make this work? If I change the server to consume the whole input and throw it away, it works, but it feels like wasting resources (some of the files being uploaded may be videos weighing hundreds of megabytes). Is there any way to change the client code to deal with that scenario?

urlconnection http java
1 Answers

You could break that call in to several requests assuming that files are big enough and making multiple requests consumes far less resources than transferring a partial file.

enum UploadStatus {
   INITIALIZED,
   STARTED,
   UPLOADED,
   ERROR
}

My Suggestion:

  1. Have a static map ConcurrentMap<File name string, UploadStatus> (or DB entry) where you can keep track of file upload statuses
  2. Create an endpoint to check and set file status
  3. Client first make a request to above endpoint
    • if file exist on the map and it's status is not UploadStatus.ERROR, set the file's status on the map to UploadStatus.INITIALIZED and let client (client A) know it can upload the file (Should do this on a synchronized block)
  4. If file exists and UploadStatus.INITIALIZED, let that client (client B) know its being uploaded. For the sake of UX, you could make the client B poll for the file status until UploadStatus becomes ERROR or UPLOADED and then take appropriate action. i.e.
    • Re-upload file on UploadStatus.ERROR
    • Show uploaded message on UploadStatus.UPLOADED
  5. Once the server receive the request to upload the actual file from the client A, keep the file upload status up to date so that on error other clients such as Client B can re-upload a failed file.

Doing the file status check and set on a single sync block is important to avoid race condition when setting correct file status. Also, that enum is just to explain the general high level steps. Since you already have Guava, you could use Guava Cache with time base eviction for storing the file statues.

user_14311840
Related