HTTP GET with request body

Viewed 1890704

I'm developing a new RESTful webservice for our application.

When doing a GET on certain entities, clients can request the contents of the entity. If they want to add some parameters (for example sorting a list) they can add these parameters in the query string.

Alternatively I want people to be able to specify these parameters in the request body. HTTP/1.1 does not seem to explicitly forbid this. This will allow them to specify more information, might make it easier to specify complex XML requests.

My questions:

  • Is this a good idea altogether?
  • Will HTTP clients have issues with using request bodies within a GET request?

https://www.rfc-editor.org/rfc/rfc2616

23 Answers

Roy Fielding's comment about including a body with a GET request.

Yes. In other words, any HTTP request message is allowed to contain a message body, and thus must parse messages with that in mind. Server semantics for GET, however, are restricted such that a body, if any, has no semantic meaning to the request. The requirements on parsing are separate from the requirements on method semantics.

So, yes, you can send a body with GET, and no, it is never useful to do so.

This is part of the layered design of HTTP/1.1 that will become clear again once the spec is partitioned (work in progress).

....Roy

Yes, you can send a request body with GET but it should not have any meaning. If you give it meaning by parsing it on the server and changing your response based on its contents, then you are ignoring this recommendation in the HTTP/1.1 spec, section 4.3:

...if the request method does not include defined semantics for an entity-body, then the message-body SHOULD be ignored when handling the request.

And the description of the GET method in the HTTP/1.1 spec, section 9.3:

The GET method means retrieve whatever information ([...]) is identified by the Request-URI.

which states that the request-body is not part of the identification of the resource in a GET request, only the request URI.

Update

The RFC2616 referenced as "HTTP/1.1 spec" is now obsolete. In 2014 it was replaced by RFCs 7230-7237. Quote "the message-body SHOULD be ignored when handling the request" has been deleted. It's now just "Request message framing is independent of method semantics, even if the method doesn't define any use for a message body" The 2nd quote "The GET method means retrieve whatever information ... is identified by the Request-URI" was deleted. - From a comment

From the HTTP 1.1 2014 Spec:

A payload within a GET request message has no defined semantics; sending a payload body on a GET request might cause some existing implementations to reject the request.

While you can do that, insofar as it isn't explicitly precluded by the HTTP specification, I would suggest avoiding it simply because people don't expect things to work that way. There are many phases in an HTTP request chain and while they "mostly" conform to the HTTP spec, the only thing you're assured is that they will behave as traditionally used by web browsers. (I'm thinking of things like transparent proxies, accelerators, A/V toolkits, etc.)

This is the spirit behind the Robustness Principle roughly "be liberal in what you accept, and conservative in what you send", you don't want to push the boundaries of a specification without good reason.

However, if you have a good reason, go for it.

You will likely encounter problems if you ever try to take advantage of caching. Proxies are not going to look in the GET body to see if the parameters have an impact on the response.

Elasticsearch accepts GET requests with a body. It even seems that this is the preferred way: Elasticsearch guide

Some client libraries (like the Ruby driver) can log the cry command to stdout in development mode and it is using this syntax extensively.

What you're trying to achieve has been done for a long time with a much more common method, and one that doesn't rely on using a payload with GET.

You can simply build your specific search mediatype, or if you want to be more RESTful, use something like OpenSearch, and POST the request to the URI the server instructed, say /search. The server can then generate the search result or build the final URI and redirect using a 303.

This has the advantage of following the traditional PRG method, helps cache intermediaries cache the results, etc.

That said, URIs are encoded anyway for anything that is not ASCII, and so are application/x-www-form-urlencoded and multipart/form-data. I'd recommend using this rather than creating yet another custom json format if your intention is to support ReSTful scenarios.

GET, with a body!?

Specification-wise you could, but, it's not a good idea to do so injudiciously, as we shall see.

RFC 7231 §4.3.1 states that a body "has no defined semantics", but that's not to say it is forbidden. If you attach a body to the request and what your server/app makes out of it is up to you. The RFC goes on to state that GET can be "a programmatic view on various database records". Obviously such view is many times tailored by a large number of input parameters, which are not always convenient or even safe to put in the query component of the request-target.

The good: I like the verbiage. It's clear that one read/get a resource without any observable side-effects on the server (the method is "safe"), and, the request can be repeated with the same intended effect regardless of the outcome of the first request (the method is "idempotent").

The bad: An early draft of HTTP/1.1 forbade GET to have a body, and - allegedly - some implementations will even up until today drop the body, ignore the body or reject the message. For example, a dumb HTTP cache may construct a cache key out of the request-target only, being oblivious to the presence or content of a body. An even dumber server could be so ignorant that it treats the body as a new request, which effectively is called "request smuggling" (which is the act of sending "a request to one device without the other device being aware of it" - source).

Due to what I believe is primarily a concern with inoperability amongst implementations, work in progress suggests to categorize a GET body as a "SHOULD NOT", "unless [the request] is made directly to an origin server that has previously indicated, in or out of band, that such a request has a purpose and will be adequately supported" (emphasis mine).

The fix: There's a few hacks that can be employed for some of the problems with this approach. For example, body-unaware caches can indirectly become body-aware simply by appending a hash derived from the body to the query component, or disable caching altogether by responding a cache-control: no-cache header from the server.

Alas when it comes to the request chain, one is often not in control of- or even aware, of all present and future HTTP intermediaries and how they will deal with a GET body. That's why this approach must be considered generally unreliable.

But POST, is not idempotent!

POST is an alternative. The POST request usually includes a message body (just for the record, body is not a requirement, see RFC 7230 §3.3.2). The very first use case example from RFC 7231 (§4.3.3) is "providing a block of data [...] to a data-handling process". So just like GET with a body, what happens with the body on the back-end side is up to you.

The good: Perhaps a more common method to apply when one wish to send a request body, for whatever purpose, and so, will likely yield the least amount of noise from your team members (some may still falsely believe that POST must create a resource).

Also, what we often pass parameters to is a search function operating upon constantly evolving data, and a POST response is only cacheable if explicit freshness information is provided in the response.

The bad: POST requests are not defined as idempotent, leading to request retry hesitancy. For example, on page reload, browsers are unwilling to resubmit an HTML form without prompting the user with a nonreadable cryptic message.

The fix: Well, just because POST is not defined to be idempotent doesn't mean it mustn't be. Indeed, RFC 7230 §6.3.1 writes: "a user agent that knows (through design or configuration) that a POST request to a given resource is safe can repeat that request automatically". So, unless your client is an HTML form, this is probably not a real problem.

QUERY is the holy grail

There's a proposal for a new method QUERY which does define semantics for a message body and defines the method as idempotent. See this.

Edit: As a side-note, I stumbled into this StackOverflow question after having discovered a codebase where they solely used PUT requests for server-side search functions. This were their idea to include a body with parameters and also be idempotent. Alas the problem with PUT is that the request body has very precise semantics. Specifically, the PUT "requests that the state of the target resource be created or replaced with the state [in the body]" (RFC 7231 §4.3.4). Clearly, this excludes PUT as a viable option.

I wouldn't advise this, it goes against standard practices, and doesn't offer that much in return. You want to keep the body for content, not options.

You have a list of options which are far better than using a request body with GET.

Let' assume you have categories and items for each category. Both to be identified by an id ("catid" / "itemid" for the sake of this example). You want to sort according to another parameter "sortby" in a specific "order". You want to pass parameters for "sortby" and "order":

You can:

  1. Use query strings, e.g. example.com/category/{catid}/item/{itemid}?sortby=itemname&order=asc
  2. Use mod_rewrite (or similar) for paths: example.com/category/{catid}/item/{itemid}/{sortby}/{order}
  3. Use individual HTTP headers you pass with the request
  4. Use a different method, e.g. POST, to retrieve a resource.

All have their downsides, but are far better than using a GET with a body.

What about nonconforming base64 encoded headers? "SOMETHINGAPP-PARAMS:sdfSD45fdg45/aS"

Length restrictions hm. Can't you make your POST handling distinguish between the meanings? If you want simple parameters like sorting, I don't see why this would be a problem. I guess it's certainty you're worried about.

Even if a popular tool use this, as cited frequently on this page, I think it is still quite a bad idea, being too exotic, despite not forbidden by the spec.

Many intermediate infrastructures may just reject such requests.

By example, forget about using some of the available CDN in front of your web site, like this one:

If a viewer GET request includes a body, CloudFront returns an HTTP status code 403 (Forbidden) to the viewer.

And yes, your client libraries may also not support emitting such requests, as reported in this comment.

An idea on an old question:

Add the full content on the body, and a short hash of the body on the querystring, so caching won't be a problem (the hash will change if body content is changed) and you'll be able to send tons of data when needed :)

If you want to allow a GET request with a body, a way is to support POST request with header "X-HTTP-Method-Override: GET". It is described here : https://en.wikipedia.org/wiki/List_of_HTTP_header_fields. This header means that while the method is POST, the request should be treated as if it is a GET. Body is allowed for POST, so you're sure nobody willl drop the payload of your GET requests.

This header is oftenly used to make PATCH or HEAD requests through some proxies that do not recognize those methods and replace them by GET (always fun to debug!).

Create a Requestfactory class

import java.net.URI;

import javax.annotation.PostConstruct;

import org.apache.http.client.methods.HttpEntityEnclosingRequestBase;
import org.apache.http.client.methods.HttpUriRequest;
import org.springframework.http.HttpMethod;
import org.springframework.http.client.HttpComponentsClientHttpRequestFactory;
import org.springframework.stereotype.Component;
import org.springframework.web.client.RestTemplate;

@Component
public class RequestFactory {
    private RestTemplate restTemplate = new RestTemplate();

    @PostConstruct
    public void init() {
        this.restTemplate.setRequestFactory(new HttpComponentsClientHttpRequestWithBodyFactory());
    }

    private static final class HttpComponentsClientHttpRequestWithBodyFactory extends HttpComponentsClientHttpRequestFactory {
        @Override
        protected HttpUriRequest createHttpUriRequest(HttpMethod httpMethod, URI uri) {
            if (httpMethod == HttpMethod.GET) {
                return new HttpGetRequestWithEntity(uri);
            }
            return super.createHttpUriRequest(httpMethod, uri);
        }
    }

    private static final class HttpGetRequestWithEntity extends HttpEntityEnclosingRequestBase {
        public HttpGetRequestWithEntity(final URI uri) {
            super.setURI(uri);
        }

        @Override
        public String getMethod() {
            return HttpMethod.GET.name();
        }
    }

    public RestTemplate getRestTemplate() {
        return restTemplate;
    }
}

and @Autowired where ever you require and use, Here is one sample code GET request with RequestBody

 @RestController
 @RequestMapping("/v1/API")
public class APIServiceController {
    
    @Autowired
    private RequestFactory requestFactory;
    

    @RequestMapping(method = RequestMethod.GET, path = "/getData")
    public ResponseEntity<APIResponse> getLicenses(@RequestBody APIRequest2 APIRequest){
        APIResponse response = new APIResponse();
        HttpHeaders headers = new HttpHeaders();
        headers.setContentType(MediaType.APPLICATION_JSON);
        Gson gson = new Gson();
        try {
            StringBuilder createPartUrl = new StringBuilder(PART_URL).append(PART_URL2);
            
            HttpEntity<String> entity = new HttpEntity<String>(gson.toJson(APIRequest),headers);
            ResponseEntity<APIResponse> storeViewResponse = requestFactory.getRestTemplate().exchange(createPartUrl.toString(), HttpMethod.GET, entity, APIResponse.class); //.getForObject(createLicenseUrl.toString(), APIResponse.class, entity);
    
            if(storeViewResponse.hasBody()) {
                response = storeViewResponse.getBody();
            }
            return new ResponseEntity<APIResponse>(response, HttpStatus.OK);
        }catch (Exception e) {
            e.printStackTrace();
            return new ResponseEntity<APIResponse>(response, HttpStatus.INTERNAL_SERVER_ERROR);
        }
        
    }
}
Related