python scrapy won't extract <script> tag

I'm new to Scrapy and my code is not able to extract the desired script tag from the HTML responses.

The HTML response for every request starts with the following structure, as shown in the browser developer tools (example from a random response, with the rest of the HTML omitted, as the relevant data is always on the same script tag):

<html  xmlns="http://www.w3.org/1999/xhtml"><head><script src="/pje/a4j/g/3_3_3.Final/org/ajax4jsf/framework.pack.js" type="text/javascript"></script><script src="/pje/a4j/g/3_3_3.Final/org/richfaces/ui.pack.js" type="text/javascript"></script><link class="component" href="/pje/a4j/s/3_3_3.Finalorg/richfaces/renderkit/html/css/basic_classes.xcss/DATB/eAELXT5DOhSIAQ!sA18_" rel="stylesheet" type="text/css" /><link class="component" href="/pje/a4j/s/3_3_3.Finalorg/richfaces/renderkit/html/css/extended_classes.xcss/DATB/eAELXT5DOhSIAQ!sA18_" media="rich-extended-skinning" rel="stylesheet" type="text/css" /><link class="component" href="/pje/a4j/s/3_3_3.Final/org/richfaces/skin.xcss/DATB/eAELXT5DOhSIAQ!sA18_" rel="stylesheet" type="text/css" /><script id="org.ajax4jsf.queue_script" type="text/javascript">if (typeof A4J != 'undefined') { if (A4J.AJAX) { with (A4J.AJAX) {if (!EventQueue.getQueue('org.richfaces.queue.global')) { EventQueue.addQueue(new EventQueue('org.richfaces.queue.global',null,null)) };}}};</script><script type="text/javascript">window.RICH_FACES_EXTENDED_SKINNING_ON=true;</script><link class="user" href="/pje/stylesheet/estilos/bootstrap/bootstrap.min.css" rel="stylesheet" type="text/css" /><link class="user" href="/pje/stylesheet/dropzone/dropzone.css" rel="stylesheet" type="text/css" /><link class="user" href="/pje/stylesheet/estilos/richfaces/tema.css" rel="stylesheet" type="text/css" /><link class="user" href="/pje/stylesheet/padrao.css" rel="stylesheet" type="text/css" /><link class="user" href="/pje/stylesheet/autos-digitais.css" rel="stylesheet" type="text/css" /><script src="/pje/js/modernizr.custom.js" type="text/javascript"></script><script src="/pje/js/jquery-2.1.4.min.js" type="text/javascript"></script><script src="/pje/js/bootstrap/bootstrap.min.js" type="text/javascript"></script><script src="/pje/js/jquery.maskedinput.min.js" type="text/javascript"></script><script src="/pje/js/mousetrap/mousetrap.min.js" type="text/javascript"></script><script src="/pje/js/mousetrap/plugins/global-bind/mousetrap-global-bind.js" type="text/javascript"></script><script src="/pje/js/pje/menu.js" type="text/javascript"></script><script src="/pje/js/global.js" type="text/javascript"></script><script src="/pje/js/pje/autos-digitais.js" type="text/javascript"></script><link class="user" href="/pje/stylesheet/estilos/icomoon/style.css" rel="stylesheet" type="text/css" /><script src="/pje/js/jquery.maskMoney.js" type="text/javascript"></script><script src="/pje/js/pje.js" type="text/javascript"></script><script src="/pje/js/pjeOffice.js" type="text/javascript"></script><script src="/pje/js/signerApplet.js" type="text/javascript"></script></head><script>window.open('https://api-pjestorage.tjdft.jus.br/2021063010s/0709994-47.2021.8.07.0020-1625061173643-2414698-processo.pdf?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=minio-pje%2F20210630%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20210630T135253Z&X-Amz-Expires=120&X-Amz-SignedHeaders=host&X-Amz-Signature=3348dc1ce55f1306d4555fb04f933af24ce5fa0b9c2540f5493a04bc83143be5');</script>
<?xml version="1.0" encoding="ISO-8859-1"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd" >
<html xmlns="http://www.w3.org/1999/xhtml" xmlns:c="http://java.sun.com/jsp/jstl/core">

    <head>
        
    
    <title>0709994-47.2021.8.07.0020 &middot; Processo Judicial Eletr&ocirc;nico - 1&ordm; Grau</title>

My goal is to extract the URL that is always contained in this part of the HTML response, which in the example above is: https://api-pjestorage.tjdft.jus.br/2021063010s/0709994-47.2021.8.07.0020-1625061173643-2414698-processo.pdf?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=minio-pje%2F20210630%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20210630T135253Z&X-Amz-Expires=120&X-Amz-SignedHeaders=host&X-Amz-Signature=3348dc1ce55f1306d4555fb04f933af24ce5fa0b9c2540f5493a04bc83143be5

I used response.css("script").extract() as a test, intending to use the re module for extracting the URL from the mentioned script tag, but, for some reason, Scrapy skips the desired tag and yields the following (i omitted the rest of the list, since i'm interested only in extracting the mentioned URL, but Scrapy goes on to the next script tag, skipping the one i need):

'<script src="/pje/a4j/g/3_3_3.Final/org/ajax4jsf/framework.pack.js" type="text/javascript"></script>', '<script src="/pje/a4j/g/3_3_3.Final/org/richfaces/u
i.pack.js" type="text/javascript"></script>', '<script id="org.ajax4jsf.queue_script" type="text/javascript">if (typeof A4J != \'undefined\') { if (A4J.AJAX) { with (
A4J.AJAX) {if (!EventQueue.getQueue(\'org.richfaces.queue.global\')) { EventQueue.addQueue(new EventQueue(\'org.richfaces.queue.global\',null,null)) };}}};</script>',
 '<script type="text/javascript">window.RICH_FACES_EXTENDED_SKINNING_ON=true;</script>', '<script src="/pje/js/modernizr.custom.js" type="text/javascript"></script>',
 '<script src="/pje/js/jquery-2.1.4.min.js" type="text/javascript"></script>', '<script src="/pje/js/bootstrap/bootstrap.min.js" type="text/javascript"></script>', '<
script src="/pje/js/jquery.maskedinput.min.js" type="text/javascript"></script>', '<script src="/pje/js/mousetrap/mousetrap.min.js" type="text/javascript"></script>',
 '<script src="/pje/js/mousetrap/plugins/global-bind/mousetrap-global-bind.js" type="text/javascript"></script>', '<script src="/pje/js/pje/menu.js" type="text/javasc
ript"></script>', '<script src="/pje/js/global.js" type="text/javascript"></script>', '<script src="/pje/js/pje/autos-digitais.js" type="text/javascript"></script>',
'<script src="/pje/js/jquery.maskMoney.js" type="text/javascript"></script>', '<script src="/pje/js/pje.js" type="text/javascript"></script>', '<script src="/pje/js/p
jeOffice.js" type="text/javascript"></script>', '<script src="/pje/js/signerApplet.js" type="text/javascript"></script>', '<script type="text/javascript">\n\t//<![CDA
TA[\n\t(function($){\n\t\t  $(document).ready(function() {\n\t\t\tvar selector = \'dtInicioInputDate\';\n\n\t\t\t//Seleciona elemento por id\n\t\t\tvar $input = $("in
put[id$=\'" + selector + "\']");\n\t\t\t\n\t\t\tif($input.length < 1){\n\t\t\t\t//Seleciona elemento por class\n\t\t\t\t$input = $("input" + selector);\n\t\t\t}\n\t\t
\t\n\t\t\tif (\'99/99/9999\' == \'\') {\n\t\t\t\t$input.unmask();\n\t\t\t} else {\n\t\t\t\t$input.mask(\'99/99/9999\');\n\t\t\t}\n\t\t });\n\t})(jQuery_21);\n\t//]]>\
n\t</script>'

The last element on the list above appears after the desired script tag (omitted from the HTML response, as it's not relevant for the task):

'<script type="text/javascript">\n\t//<![CDATA[\n\t(function($){\n\t\t  $(document).ready(function() {\n\t\t\tvar selector = \'dtInicioInputDate\';\n\n\t\t\t//Seleciona elemento por id\n\t\t\tvar $input = $("in
put[id$=\'" + selector + "\']");\n\t\t\t\n\t\t\tif($input.length < 1){\n\t\t\t\t//Seleciona elemento por class\n\t\t\t\t$input = $("input" + selector);\n\t\t\t}\n\t\t
\t\n\t\t\tif (\'99/99/9999\' == \'\') {\n\t\t\t\t$input.unmask();\n\t\t\t} else {\n\t\t\t\t$input.mask(\'99/99/9999\');\n\t\t\t}\n\t\t });\n\t})(jQuery_21);\n\t//]]>\
n\t</script>'

I also tried using

pattern = re.compile(r"'(https://api-pjestorage.tjdft.jus.br/.+)'")
lst_find = re.findall(pattern=pattern, string=response.text)

But it returns an empty list, even though the pattern works correctly when i copy the HTML as a string and try it, which suggests that the desired script tag is not contained in "response.text" for some reason that i don't understand.

How can i get the full raw HTML text for using regex or how can i make sure that the response.css (or xpath) will extract the desired script tag?

And why Scrapy is skipping one script tag but extracting all the others correctly?

Unfortunately, i can't share the page i'm trying to scrape, since a login and password are needed.

Any suggestion will be greatly appreciated. Sorry for the poor english.