differential update of pyinstaller executable (modify embedded PYZ-00.pyz)

This is quite a complex problem, but I think this may be at least part of what you're looking for.

Based on your example, I changed prog.py so it imports pretty normally when running from source, but when frozen with pyinstaller runs from pyc files directly.

import sys

def import_pyc(name):
    import py_compile
    import types
    import marshal
    
    pyversion = f"{sys.version_info.major}{sys.version_info.minor}"
    filename = f"{name}.cpython-{pyversion}.pyc"
    
    with open(filename, "rb") as pyc_file:
        # pyc files have 16 bytes reserved at the start in python 3.7+
        # due to https://www.python.org/dev/peps/pep-0552/
        # may change again in the future
        pyc_file.seek(16) 
        code_obj = marshal.load(pyc_file)

    module = types.ModuleType(name)
    exec(code_obj, module.__dict__)

    globals()[name] = module

def import_py(name):
    import importlib
    
    globals()[name] = importlib.import_module("mod1")
    
def import2(name):
    if getattr(sys, "frozen", False):
        import_pyc(name)
    else:
        import_py(name)


import2("mod1")

if __name__ == "__main__":
    mod1.hello()

This is heavily based on the wonderful answer here.

This means that mod.py is not packaged by PyInstaller, you will have to include mod1.cpython-38.pyc as a data file.

One convenient way to do this is with the command PyInstaller --add-data "__pycache__/*;." prog.py (Although switch the semicolon for a colon if you're not on Windows). This puts everything in the __pycache__ folder, all your imported modules, into your ending dist/prog folder. Note that if you run this multiple times PyInstaller puts a pyc for the main python folder in __pycache__, so that will get bundled on subsequent runs.

Depending on how you bundle and run your project, you will probably run into problems where the current working directory is off, which will result in a FileNotFound when you try to load the pycs. I can't give you a silver bullet to find the path you want, since it depends on how you end up doing things, but ways I commonly use to find the absolute path that should be the current working directory are os.path.dirname(sys.executable) and os.path.dirname(os.path.abspath(__file__)).

This solution is neither capable of 'patching' a .PYZ file nor capable of putting all .pyc files into a zip file.

But so far it is the only viable solution I found so far, that works for huge projects with loads of third party dependencies.

The idea is to remove all (or most files from the .PYZ file) and copy the corresponding .pyc files into the working directory.

I will enhance and elaborate this answer over time. I'm still experimenting:

I achieve this by modyfing the spec file:

• determine the directory MYDIR where the spec file is located in
• create a directory, MYDIR/src where all the files from a.pure shall be copied to
• copy all files from a.pure to to MYDIR/src. (with subdirectories corresponding to the module's name. Module mypackage.mod.common would for example be stored in MYDIR/src/mypackage/mod/common.py)
• iterate through files and compile them to a .pyc file and remove .py file afterwards.
• create a PYZ file which contains only the files that are not copied. (in my test case, keep no .pyc file in the PYZ)
• create exe with the modified PYZ
• collect all files that should be collected plus also all files from MYDIR/src (e.g. with a.datas + Tree("src")

Spec file Changes: At the beginning

import os
MYDIR = os.path.realpath(SPECPATH)
sys.path.append(MYDIR)
import mypyinsthelpers  # allows to reuse the code in multiple projects

Then after the (unmodified) a = Analysis(... section I add.

to_rmv_from_pyc = mypyinsthelpers.mk_copy_n_compile(a.pure, MYDIR)

# modified creation of pyz`
pyz = PYZ(a.pure - to_rmv_from_pyc, a.zipped_data,
             cipher=block_cipher)

I will detail the function mypyinsthelpers.mk_copy_n_compile further down

Change the collect phase:

Instead of

coll = COLLECT(exe,
               a.binaries,
               a.zipfiles,
               a.datas,
...

I write:

coll = COLLECT(exe,
               a.binaries,
               a.zipfiles,
               a.datas + Tree("src"),
...

And here the declaration of mypyinsthelpers.mk_copy_n_compile()

import compileall
import os
import shutil
from pathlib import Path


def mk_copy_n_compile(toc, src_tree):
    """
    - copy source files to a destination directory
    - compile them as pyc
    - delete source
    """
    dst_base_path = os.path.join(src_tree, "src")
    to_rm = []
    # copy files to destination tree
    for entry in toc:
        modname, src, typ = entry
        assert typ == "PYMODULE"
        assert src.endswith(".py") or src.endswith(".pyw")
        # TODO: might add logic to skip some files (keep them in PYC)
        to_rm.append(entry)

        if src.endswith("__init__.py"):
            modname += ".__init__"

        m_split = modname.split(".")
        m_split[-1] += ".py"
        dst_dir = os.path.join(dst_base_path, *m_split[:-1])
        dst_path = os.path.join(dst_dir, m_split[-1])
        if not os.path.isdir(dst_dir):
            os.makedirs(dst_dir)
        print(entry[:2], dst_path)
        shutil.copy(src, dst_path)

    # now compile all files and rmv src
    top_tree = src_tree
    src_tree = os.path.join(src_tree, "src")
    curdir = os.getcwd()
    os.chdir(dst_base_path)
    for path in Path(dst_base_path).glob("**/*.py"):
        # TODO: might add code to keep some files as source
        compileall.compile_file(
            str(path.relative_to(dst_base_path)), quiet=1, legacy=True)
        path.unlink()
    os.chdir(curdir)
    return to_rm