The *html files of rendered vignettes are too big for CRAN

I'm cleaning up an R package in order to submit to CRAN, and I have the following note:

* checking installed package size ... NOTE
  installed size is 11.3Mb
  sub-directories of 1Mb or more:
    doc  10.1Mb

I invested a great deal of time in making three vignettes as R Markdown files *.Rmd, and it appears that the resulting HTML files are too large for CRAN. I'm reluctant to simply delete these, as it would be useful for these vignettes to be easily accessible for CRAN users.

Here is the header I'm using for the vignettes:

---
title: "The Title of the Vignettes"
output: 
  rmarkdown::html_vignette:
    toc: true
vignette: >
  %\VignetteIndexEntry{"The Title of the Vignettes"}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

The knitr::rmarkdown files are not a problem, but the *html files are too large.

What is the correct approach here? I don't think I can simply make R ignore these files via .Rbuildignore.

Are there other examples where this has been solved?