I'm trying to make calls to the Census microdata API, but having trouble with URL formatting in the recommended libraries (Drakma and Dexador). in order to return tabulated results, the census API calls for non-standard (maybe?) encoding: queries with '+' should not be switched to '%2B', while the 'tabulate commands' are encoded entirely to % encoding. EG:
https:.../acs5/pums?tabulate=weight(PWGTP)&col+AGEGROUP&recode+AGEGROUP="tabulate commands here"
;;; where "tabulate commands" look like {"b":AGEP,"d":[[0,1,2,3],[{"mn":4,"mx":99}]]} before encoding
;;; and alphabet soup after.
If I wrap the above in Drakma or Dexador/Quri to make the http request, it encodes the entire URL, with the encoded "tabulate commands" ("%7B%22b%22%3A%22AGEP%22..." etc.) being accepted by the census API, but creating "...col%2BAGEGROUP..." and "...recode%2BAGEGROUP..." as well, which the census API chokes on (returns a bad query error).
If I manually edit the output URL back to '..col+AGEGROUP..' and '...recode+AGEGROUP...' and paste it into a browser, it works. So, is there a way to prevent the '+' characters from transforming? I'm out of my depth with web work in general, so I apologize if this question is unclear.
Straightforward calls work fine entirely encoded; it seems to be specifically the '+' named variables involved in tabulate queries that cause trouble. (Beware, this example pulls a lot of entries):
(dex:get (quri:make-uri
:defaults "https://api.census.gov/data/2019/acs/acs5/pums"
:query '(("get" . "NP")
("for" . "state:01"))))
(Side note -- it seems strange that I can't pass parameters/queries directly to the 'GET' method for Dexador, like I can in Drakma. Am I missing something obvious?)
Alternately, I can roll my own encoder, and only encode the portion I need encoded ('Recode commands here'), and then pass dex the URL as a string -- this works! but seems fragile because I'm not encoding most of the query. This is fine for my actual use, because I mostly need to repeat static calls and only vary the geography, just seems like bad practice, and I'm trying to figure out a more general approach.
Another side note: I switched to Dex from Drakma because I couldn't get Drakma to process a plain string -- it always encoded what I passed it, and I couldn't seem to turn that off.