How to retrieve Collection items from Internet archive

user_12351892 0
Viewed 408

Is there an api to retrieve the items of a specific collection (i.e. children collections)?

Example: The HTML for the collection movies can be retrieved from https://archive.org/details/movies and JSON/meta from https://archive.org/metadata/movies/ . However the JSON payload has no reference to the child items of the collection (i.e. the sub-collections such "Arts & Music", "Community videos" etc).

I'm trying to build a wrapper around internet archive API. The API information is quite sparse[0][1][2] and incomplete.

Note: please create a 'internet-archive' tag if you have a reputation higher than 1500

[0] http://blog.archive.org/2011/03/31/how-archive-org-items-are-structured/ [1] http://blog.archive.org/developers/

[2] https://archive.org/services/docs/api/index.html

web-crawler
1 Answers

After writing this response I realised it's not really an answer to your question because it doesn't return sub-collections but all items instead. I edited my answer slightly to reflect that and I am leaving it up for future reference.

I struggled with just listing all items for a collection - I tried using the Simple Lists API but unlike the library_of_atlantis example from the docs, it wouldn't return any results for another collection I was interested in.

Eventually I got a helpful response from the IA team suggesting to make an advanced search query such as this one: https://archive.org/advancedsearch.php?q=collection:movies&fl[]=identifier&rows=100&page=1&output=json.

There might be a better way to find all the sub-collections of the movies collection, but if you add the fl[]=collection parameter to the query, it will return a list of all the collections each of the items is in. Maybe you could collect the collection names from there?

As for why the Simple Lists API wouldn't work: apparently not all collections are populated by simplelist (while the presence of a *_simplelists.sqlite https://archive.org/download/library_of_atlantis seems to indicate that collection indeed uses simplelist).

user_13346880
Related