How to extract the file modification time of a scraped image?

user_1147688 0
Viewed 673

I'm trying to scrape part of a part-website that contain images of the parts, to collect some statistics. However, there is no url or image upload or creation date, so I have to use the approximate image file modification-date to get this info. Using cURL, this is an easy task with:

curl -sI https://path.to.com/blahblah_123/item_picture.jpg |grep "last-modified"

However, I think it would be more convenient to get this within the scrapy spider. But I have no idea if scrapy supports this at all, since I cannot find it in the documentation.

Is there a way to get the last-modified date of a scraped image in scrapy?

scrapy-spider scrapy web-scraping python
1 Answers

From the documentation of Scrapy, the response has a headers dict field.

So you can access the last-modified with response.headers.get('Last-Modified').

user_1228610
Related