Pyinstaller Jinja2 TemplateNotFound

I am using pyinstaller to build my flask application, everything is working fine except I get problems with Jinja2 templates.

It gave me jinja2.exceptions.TemplateNotFound,

I tried to put from app import template which is the templates folder, but it didn't work (I guess since they don't contain any py file).

I also tried changing the .spec file to include the templates folder

added_files = [
         ( '..\\CommerceApp\\app\\templates', 'templates' ),
         ( '..\\CommerceApp\\app\\static', 'static' )
        ]

a = Analysis(['..\\CommerceApp\\run.py'],
             pathex=['D:\\PythonProjects\\CommerceAppExe'],
             binaries=None,
             datas=added_files,
             hiddenimports=[],
             hookspath=[],
             runtime_hooks=[],
             excludes=[],
             win_no_prefer_redirects=False,
             win_private_assemblies=False,
             cipher=block_cipher)

But it didn't work either, same result as if I copy the folder manually by myself.

Is there any way to include Template bundled together with the .exe?

Edit

This is my spec file

# -*- mode: python -*-

block_cipher = None

a = Analysis(['..\\CommerceApp_withPyInstaller\\run.py'],
             pathex=['D:\\PythonProjects\\CommerceAppExe'],
             binaries=None,
             datas=[],
             hiddenimports=[],
             hookspath=[],
             runtime_hooks=[],
             excludes=[],
             win_no_prefer_redirects=False,
             win_private_assemblies=False,
             cipher=block_cipher)
pyz = PYZ(a.pure, a.zipped_data,
             cipher=block_cipher)
exe = EXE(pyz,
          a.scripts,
          exclude_binaries=True,
          name='SupplyTracker',
          debug=False,
          strip=False,
          upx=True,
          console=True )
coll = COLLECT(exe,
               a.binaries,
               a.zipfiles,
               a.datas,
               strip=False,
               upx=True,
               name='SupplyTracker')

Edit 2

Accepted Answer changed to gmas80 because it fixes the problem.

Edit 3

Also I just realize, I can just make a new folder with my package name and fill in the static templates css, html, etc, and it is gonna work (similar result from what gmas80 script does)