Python PIL convert tif to pdf - issues with file size, quality

I am using Python PIL Image to convert tif files to pdf, but I am getting changes in file size and quality that I can't seem to manage.

The code was adapted from here. (Python 3.x)

def tiff_to_pdf(tiff_path):

pdf_path = tiff_path + '.pdf'
if not os.path.exists(tiff_path): raise Exception(f'{tiff_path} does not find.')
image = Image.open(tiff_path)

images = []
for i, page in enumerate(ImageSequence.Iterator(image)):
    page = page.convert("RGB")
    images.append(page)
if len(images) == 1:
    images[0].save(pdf_path, quality = 85, optimize=True)
else:
    images[0].save(pdf_path, quality = 85, optimize=True, save_all=True, append_images=images[1:])

os.remove(tiff_path)
os.rename(pdf_path, tiff_path)

First I tested on image tif's (photos ~2MB), and found a reduction in file size of 90%, and very little loss in quality, this wasn't an issue. The clients tif's are actually scanned documents (~45kb) including lots of handwriting though. I tested on some of these and found the file size went up by 1200% and had a slight loss in quality, mostly I am seeing some very light grey pixels around the edges of letters where it was originally pure white.

I've experimented with the save parameters quality, optimize, and resolution. Resolution had no affect. Increasing quality from the default 75 to 85 helped a bit, and setting optimize=True helped with file size slightly. Even dropping quality to 5 is resulting in filesize growing to 300%, and terrible image quality.

The increase in file size is likely not going to be acceptable since there are 355GB of tifs which increase by about 4TB.

I'm hoping there is some setting I am overlooking that just keeps file size and quality the same as the original. I have a strong preference for this PIL module since it's in all the clients systems already. Adding a module adds a lot of extra work here, so I don't want to do that unless there is no choice.