send referrer header with chrome.downloads api

Question

I am trying to create an extension which emulates the Ctrl+Click to save image feature from Opera 12. I am using chrome.downloads.download() from my extension's background script to initiate downloads, while a content script listens for the user action and sends a message with the URL of the image to download.

Everything works fine, except that on some sites such as pixiv.net, the download is interrupted and fails. Using the webRequest API, I verified that the cookie from the active tab is being sent with the download request, but no referer header is sent. I expect that this is because the site is blocking download requests from external sites. I have not been able to verify this though, as the webRequest.onError event isn't firing on a failed download.

I cannot set the referer header myself, as it cannot be set through chrome.downloads, and webRequest.onBeforeSendHeaders cannot be used in its blocking form with a download request, so I can't add the headers later. Is there a way to start a download in the context of a tab so that it behaves like right-click > save as...?

For clarification as to how I'm starting the downloads, here is my TypeScript code stripped down to the applicable parts.

Injected script:

window.addEventListener('click', (e) => {
    if (suspended) {
        return;
    }

    if (e.ctrlKey && (e.target).nodeName === 'IMG') {
        chrome.runtime.sendMessage({
            url: (e.target).src,
            saveAs: true,
        });

        // Prevent the click from triggering links, etc.
        e.preventDefault();
        e.stopImmediatePropagation();

        // This can trigger for multiple elements, so disable it 
        // for a moment so we don't download multiple times at once
        suspended = true;
        window.setTimeout(() => suspended = false, 100);
    }
}, true);

Background script:

interface DownloadMessage {
    url: string;
    saveAs: boolean;
}

chrome.runtime.onMessage.addListener((message: DownloadMessage, sender, sendResponse) => {
    chrome.downloads.download({
        url: message.url,
        saveAs: message.saveAs,
    });
});

Update

Going off of ExpertSystem's solution below, I've found a method that mostly works. When the background script receives a request to download an image, I now compare the URL's host name against a user-configured list of sites that need the workaround. If the workaround is needed, I send a message back to the tab. The tab downloads the image using an XMLHttpRequest with responseType = 'blob'. I then use URL.createObjectURL() on the blob and pass that URL back to the background script to download. This avoids any size limitations of data URIs. Also, if the XHR request fails, I force an attempt using the standard method so that the user will at least see a failed download appear.

The code now looks something like this:

Injected Script:

// ...Original code here...

chrome.runtime.onMessage.addListener((message, sender, sendResponse) => {
    switch (message.action) {
        case 'xhr-download':
            var xhr = new XMLHttpRequest();
            xhr.responseType = 'blob';

            xhr.addEventListener('load', (e) => {
                chrome.runtime.sendMessage({
                    url: URL.createObjectURL(xhr.response),
                    filename: message.url.substr(message.url.lastIndexOf('/') + 1),
                    saveAs: message.saveAs,
                });
            });

            xhr.addEventListener('error', (e) => {
                // The XHR method failed. Force an attempt using the
                // downloads API.
                chrome.runtime.sendMessage({
                    url: message.url,
                    saveAs: message.saveAs,
                    forceDownload: true,
                });
            });

            xhr.open('get', message.url, true);
            xhr.send();
            break;
    }
});

Background Script:

interface DownloadMessage {
    url: string;
    saveAs: boolean;
    filename?: string;
    forceDownload?: boolean;
}

chrome.runtime.onMessage.addListener((message: DownloadMessage, sender, sendResponse) => {
    var downloadOptions = {
        url: message.url,
        saveAs: message.saveAs,
    };

    if (message.filename) {
        options.filename = message.filename;
    }

    var a = document.createElement('a');
    a.href = message.url;

    if (message.forceDownload || a.protocol === 'blob:' || !needsReferralWorkaround(a.hostname)) {
        // No workaround needed or the XHR workaround is giving
        // us its results.
        chrome.downloads.download(downloadOptions);

        if (a.protocol === 'blob:') {
            // We don't need the blob URL any more. Release it.
            URL.revokeObjectUrl(message.url);
        }
    } else {
        // The XHR workaround is needed. Inform the tab.
        chrome.tabs.sendMessage(sender.tab.id, { 
            action: 'xhr-download', 
            url: message.url, 
            saveAs: message.saveAs
        });
    }
});

This will, of course, fail if the image is not on the same domain as the page and the server doesn't send CORS headers, but I figure there won't be too many sites which restrict image downloads by referrer and serve images from a different domain.

send referrer header with chrome.downloads api

1 Answers