How to replace hyperlinks in StreamingResponse?

userNJwC 0
Viewed 80

Is that possible to replace hyperlinks in StreamingResponse?

I'm using below code to stream HTML content.

from starlette.requests import Request
from starlette.responses import StreamingResponse
from starlette.background import BackgroundTask

import httpx

client = httpx.AsyncClient(base_url="http://containername:7800/")


async def _reverse_proxy(request: Request):
    url = httpx.URL(path=request.url.path, query=request.url.query.encode("utf-8"))
    rp_req = client.build_request(
        request.method, url, headers=request.headers.raw, content=await request.body()
    )
    rp_resp = await client.send(rp_req, stream=True)
    return StreamingResponse(
        rp_resp.aiter_raw(),
        status_code=rp_resp.status_code,
        headers=rp_resp.headers,
        background=BackgroundTask(rp_resp.aclose),
    )


app.add_route("/titles/{path:path}", _reverse_proxy, ["GET", "POST"])

It's working fine, but I would like to replace a href links.
Is that possible?

I've tired to wrap the generator like below:

async def adjust_response(iterable):
    # Adjust hyperlinks in response.
    async for element in iterable.aiter_raw():
        yield element.decode("utf-8").replace("/admin", "/gateway/admins/SERVICE_A").encode("utf-8")

but this caused that error:

h11._util.LocalProtocolError: Too much data for declared Content-Length
httpx starlette streaming fastapi python
1 Answers

One solution would clearly be to read from the original response generator (as mentioned in the comments section above), modify each href link, and then yield the modified content.

Another solution would be to use JavaScript to find all links in the HTML document and modify them accordingly. If you had access to the external service's HTML files, you could just add a script to modify all the href links, only if the Window.location is not pointing to the service's host (e.g., if (window.location.host != "containername:7800" ) {...}). Even though you don't have access to the external HTML files, you could still do that on server side. You can create a StaticFiles instance to serve a replace.js script file, and simply inject that script using a <script> tag in the <head> section of the HTML page (Note: if no <head> tag is provided, then find the <html> tag and create the <head></head> with the <script> in it). You can have the script run when the whole page has loaded, using window.onload event, or, preferably, when the initial HTML document has been completely loaded and parsed (without waiting for stylesheets, images, etc., to finish loading) using DOMContentLoaded event. Using this approach, you don't have to go through each chunk to modify each href link on server side, but rather inject the script and then have the replacement taking place on client side. Example:

# ...
from fastapi.staticfiles import StaticFiles

app = FastAPI()
app.mount("/static-js", StaticFiles(directory="static-js"), name="static-js")

client = httpx.AsyncClient(base_url="http://containername:7800/")

async def iter_content(r):
    found = False
    async for chunk in r.aiter_raw():
        if not found:
            idx = chunk.find(bytes('<head>', 'utf-8'))
            if idx != -1:
                found = True
                b_arr = bytearray(chunk)
                b_arr[idx+6:] = bytes('<script src="/static-js/replace.js"></script>', 'utf-8') + b_arr[idx+6:]
                chunk = bytes(b_arr)
        yield chunk

async def _reverse_proxy(request: Request):
    url = httpx.URL(path=request.url.path, query=request.url.query.encode("utf-8"))
    rp_req = client.build_request(
        request.method, url, headers=request.headers.raw, content=await request.body()
    )
    rp_resp = await client.send(rp_req, stream=True)
    return StreamingResponse(
        iter_content(rp_resp),
        status_code=rp_resp.status_code,
        headers=rp_resp.headers,
        background=BackgroundTask(rp_resp.aclose),
    )

app.add_route("/titles/{path:path}", _reverse_proxy, ["GET", "POST"])

The JS script (replace.js):

document.addEventListener('DOMContentLoaded', (event) => {
   var anchors = document.getElementsByTagName("a");

   for (var i = 0; i < anchors.length; i++) {
      let path = anchors[i].pathname.replace('/admin', '/admins/SERVICE_A');
      anchors[i].href = path + anchors[i].search + anchors[i].hash;
   }
});
userKJXc10
Related