Python urllib2, basic HTTP authentication, and tr.im

Question

I'm playing around, trying to write some code to use the tr.im APIs to shorten a URL.

After reading http://docs.python.org/library/urllib2.html, I tried:

   TRIM_API_URL = 'http://api.tr.im/api'
   auth_handler = urllib2.HTTPBasicAuthHandler()
   auth_handler.add_password(realm='tr.im',
                             uri=TRIM_API_URL,
                             user=USERNAME,
                             passwd=PASSWORD)
   opener = urllib2.build_opener(auth_handler)
   urllib2.install_opener(opener)
   response = urllib2.urlopen('%s/trim_simple?url=%s'
                              % (TRIM_API_URL, url_to_trim))
   url = response.read().strip()

response.code is 200 (I think it should be 202). url is valid, but the basic HTTP authentication doesn't seem to have worked, because the shortened URL isn't in my list of URLs (at http://tr.im/?page=1).

After reading http://www.voidspace.org.uk/python/articles/authentication.shtml#doing-it-properly I also tried:

   TRIM_API_URL = 'api.tr.im/api'
   password_mgr = urllib2.HTTPPasswordMgrWithDefaultRealm()
   password_mgr.add_password(None, TRIM_API_URL, USERNAME, PASSWORD)
   auth_handler = urllib2.HTTPBasicAuthHandler(password_mgr)
   opener = urllib2.build_opener(auth_handler)
   urllib2.install_opener(opener)
   response = urllib2.urlopen('http://%s/trim_simple?url=%s'
                              % (TRIM_API_URL, url_to_trim))
   url = response.read().strip()

But I get the same results. (response.code is 200 and url is valid, but not recorded in my account at http://tr.im/.)

If I use query string parameters instead of basic HTTP authentication, like this:

   TRIM_API_URL = 'http://api.tr.im/api'
   response = urllib2.urlopen('%s/trim_simple?url=%s&username=%s&password=%s'
                              % (TRIM_API_URL,
                                 url_to_trim,
                                 USERNAME,
                                 PASSWORD))
   url = response.read().strip()

...then not only is url valid but it's recorded in my tr.im account. (Though response.code is still 200.)

There must be something wrong with my code though (and not tr.im's API), because

$ curl -u yacitus:xxxx http://api.tr.im/api/trim_url.json?url=http://www.google.co.uk

...returns:

{"trimpath":"hfhb","reference":"nH45bftZDWOX0QpVojeDbOvPDnaRaJ","trimmed":"11\/03\/2009","destination":"http:\/\/www.google.co.uk\/","trim_path":"hfhb","domain":"google.co.uk","url":"http:\/\/tr.im\/hfhb","visits":0,"status":{"result":"OK","code":"200","message":"tr.im URL Added."},"date_time":"2009-03-11T10:15:35-04:00"}

...and the URL does appear in my list of URLs on http://tr.im/?page=1.

And if I run:

$ curl -u yacitus:xxxx http://api.tr.im/api/trim_url.json?url=http://www.google.co.uk

...again, I get:

{"trimpath":"hfhb","reference":"nH45bftZDWOX0QpVojeDbOvPDnaRaJ","trimmed":"11\/03\/2009","destination":"http:\/\/www.google.co.uk\/","trim_path":"hfhb","domain":"google.co.uk","url":"http:\/\/tr.im\/hfhb","visits":0,"status":{"result":"OK","code":"201","message":"tr.im URL Already Created [yacitus]."},"date_time":"2009-03-11T10:15:35-04:00"}

Note code is 201, and message is "tr.im URL Already Created [yacitus]."

I must not be doing the basic HTTP authentication correctly (in either attempt). Can you spot my problem? Perhaps I should look and see what's being sent over the wire? I've never done that before. Are there Python APIs I can use (perhaps in pdb)? Or is there another tool (preferably for Mac OS X) I can use?

Ali_28380 · Answer

Really cheap solution:

urllib.urlopen('http://user:xxxx@api.tr.im/api')

(which you may decide is not suitable for a number of reasons, like security of the url)

Github API example:

>>> import urllib, json
>>> result = urllib.urlopen('https://personal-access-token:x-oauth-basic@api.github.com/repos/:owner/:repo')
>>> r = json.load(result.fp)
>>> result.close()

user_1020470 · Answer

Take a look at this SO post answer and also look at this basic authentication tutorial from the urllib2 missing manual.

In order for urllib2 basic authentication to work, the http response must contain HTTP code 401 Unauthorized and a key "WWW-Authenticate" with the value "Basic" otherwise, Python won't send your login info, and you will need to either use Requests, or urllib.urlopen(url) with your login in the url, or add a the header like in @Flowpoke's answer.

You can view your error by putting your urlopen in a try block:

try:
    urllib2.urlopen(urllib2.Request(url))
except urllib2.HTTPError, e:
    print e.headers
    print e.headers.has_key('WWW-Authenticate')

Python urllib2, basic HTTP authentication, and tr.im

7 Answers