spaCy: How to update doc.ents when using doc.retokenize()

I am trying to update a pre-trained model with tokens using the retokenizer. I created a pipeline in order to do this. In this pipeline, I also set "ENT_TYPE" when merging the tokens.

@Language.factory("re_tokenize")
def re_tokenize(nlp, name):
    return ReTokenize(nlp.vocab)

class ReTokenize:
    pattern = ""
    
    def __init__(self, vocab):
        self.pattern = r"[a-zA-Z0-9]+\[{0,1}[a-zA-Z0-9_]+\]{0,1}\[{0,1}[a-zA-Z0-9_]+\]{0,1}\[{0,1}[a-zA-Z0-9_]+\]{0,1}#{0,1}"
    
    def __call__(self, doc):
        spans = []

        for match in re.finditer(self.pattern, doc.text):
            start, end = match.span()
            span = doc.char_span(start, end)

            if span is not None:
                spans.append(span)

        with doc.retokenize() as retokenizer:
            for span in spans:
                retokenizer.merge(span, attrs={"ENT_TYPE": "VAR"})

        return doc

Using this pipeline, I can tokenize the words correctly. Also, the data in ent_type_ seems to be updated.

BEFORE:

# Set model
nlp = spacy.load("ja_ginza")

text = "aaa_bbbとaaa_CCCの2ﾊﾞｲﾄマップ"
text = mojimoji.zen_to_han(text).lower()

doc = nlp(text)

print([token.text for token in doc])
print([token.ent_type_ for token in doc])

['aaa', '', 'bbb', 'と', 'aaa', '', 'ccc', 'の', '2', 'ﾊﾞｲﾄ', 'ﾏｯﾌﾟ']

['Product_Other', 'Product_Other', 'Product_Other', '', 'Product_Other', 'Product_Other', 'Product_Other', '', 'N_Product', 'N_Product', 'N_Product']

AFTER:

nlp.add_pipe("re_tokenize", before="parser")
doc = nlp(text)

print([token.text for token in doc])
print([token.ent_type_ for token in doc])

['aaa_bbb', 'と', 'aaa_ccc', 'の', '2', 'ﾊﾞｲﾄ', 'ﾏｯﾌﾟ']

['VAR', '', 'VAR', '', 'N_Product', 'N_Product', 'N_Product']

However, it seems that doc.ents is not being updated:

print([ent.label_ for ent in doc.ents])

['N_Product']

How do I also update doc.ents?