__pycache__ merge conflicts not resolved by gitignore

user_14409528 0
Viewed 1112

I am trying to merge a development branch back into the master. I have run git rm '*.pyc' in both files and this is my gitignore (copied from here):

# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
*$py.class

# C extensions
*.so

# Distribution / packaging
.Python
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
share/python-wheels/
*.egg-info/
.installed.cfg
*.egg
MANIFEST

# PyInstaller
#  Usually these files are written by a python script from a template
#  before PyInstaller builds the exe, so as to inject date/other infos into it.
*.manifest
*.spec

# Installer logs
pip-log.txt
pip-delete-this-directory.txt

# Unit test / coverage reports
htmlcov/
.tox/
.nox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*.cover
*.py,cover
.hypothesis/
.pytest_cache/
cover/

# Translations
*.mo
*.pot

# Django stuff:
*.log
local_settings.py
db.sqlite3
db.sqlite3-journal

# Flask stuff:
instance/
.webassets-cache

# Scrapy stuff:
.scrapy

# Sphinx documentation
docs/_build/

# PyBuilder
.pybuilder/
target/

# Jupyter Notebook
.ipynb_checkpoints

# IPython
profile_default/
ipython_config.py

# pyenv
#   For a library or package, you might want to ignore these files since the code is
#   intended to run in multiple environments; otherwise, check them in:
# .python-version

# pipenv
#   According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
#   However, in case of collaboration, if having platform-specific dependencies or dependencies
#   having no cross-platform support, pipenv may install dependencies that don't work, or not
#   install all needed dependencies.
#Pipfile.lock

# PEP 582; used by e.g. github.com/David-OConnor/pyflow
__pypackages__/

# Celery stuff
celerybeat-schedule
celerybeat.pid

# SageMath parsed files
*.sage.py

# Environments
.env
.venv
env/
venv/
ENV/
env.bak/
venv.bak/

# Spyder project settings
.spyderproject
.spyproject

# Rope project settings
.ropeproject

# mkdocs documentation
/site

# mypy
.mypy_cache/
.dmypy.json
dmypy.json

# Pyre type checker
.pyre/

# pytype static type analyzer
.pytype/

# Cython debug symbols
cython_debug/

However, when I do

git checkout master
git merge version3ascii

(version3ascii is the name of my development branch)

I am getting the following error messages:

CONFLICT (rename/delete): venv/lib/python3.7/site-packages/astroid/brain/__pycache__/brain_subprocess.cpython-37 2.pyc deleted in version3ascii and renamed to venv/lib/python3.7/site-packages/astroid/brain/brain_subprocess 3.py in HEAD. Version HEAD of venv/lib/python3.7/site-packages/astroid/brain/brain_subprocess 3.py left in tree.
CONFLICT (rename/delete): venv/lib/python3.7/site-packages/astroid/brain/__pycache__/brain_ssl.cpython-37.pyc deleted in version3ascii and renamed to venv/lib/python3.7/site-packages/astroid/brain/brain_ssl 4.py in HEAD. Version HEAD of venv/lib/python3.7/site-packages/astroid/brain/brain_ssl 4.py left in tree.
CONFLICT (rename/delete): venv/lib/python3.7/site-packages/astroid/brain/__pycache__/brain_ssl.cpython-37 2.pyc deleted in version3ascii and renamed to venv/lib/python3.7/site-packages/astroid/brain/brain_ssl 3.py in HEAD. Version HEAD of venv/lib/python3.7/site-packages/astroid/brain/brain_ssl 3.py left in tree.
CONFLICT (rename/delete): venv/lib/python3.7/site-packages/astroid/brain/__pycache__/brain_scipy_signal.cpython-37 2.pyc deleted in version3ascii and renamed to venv/lib/python3.7/site-packages/astroid/brain/brain_scipy_signal 4.py in HEAD. Version HEAD of venv/lib/python3.7/site-packages/astroid/brain/brain_scipy_signal 4.py left in tree.
CONFLICT (rename/delete): venv/lib/python3.7/site-packages/astroid/brain/__pycache__/brain_scipy_signal 2.cpython-37.pyc deleted in version3ascii and renamed to venv/lib/python3.7/site-packages/astroid/brain/brain_scipy_signal 3.py in HEAD. Version HEAD of venv/lib/python3.7/site-packages/astroid/brain/brain_scipy_signal 3.py left in tree.
CONFLICT (rename/delete): venv/lib/python3.7/site-packages/astroid/brain/__pycache__/brain_responses.cpython-37 2.pyc deleted in version3ascii and renamed to venv/lib/python3.7/site-packages/astroid/brain/brain_responses 4.py in HEAD. Version HEAD of venv/lib/python3.7/site-packages/astroid/brain/brain_responses 4.py left in tree.
CONFLICT (rename/delete): venv/lib/python3.7/site-packages/astroid/brain/__pycache__/brain_responses.cpython-37.pyc deleted in version3ascii and renamed to venv/lib/python3.7/site-packages/astroid/brain/brain_responses 3.py in HEAD. Version HEAD of venv/lib/python3.7/site-packages/astroid/brain/brain_responses 3.py left in tree.
CONFLICT (rename/delete): venv/lib/python3.7/site-packages/astroid/brain/__pycache__/brain_re.cpython-37.pyc deleted in version3ascii and renamed to venv/lib/python3.7/site-packages/astroid/brain/brain_re 4.py in HEAD. Version HEAD of venv/lib/python3.7/site-packages/astroid/brain/brain_re 4.py left in tree.
CONFLICT (rename/delete): venv/lib/python3.7/site-packages/astroid/brain/__pycache__/brain_re.cpython-37 2.pyc deleted in version3ascii and renamed to venv/lib/python3.7/site-packages/astroid/brain/brain_re 3.py in HEAD. Version HEAD of venv/lib/python3.7/site-packages/astroid/brain/brain_re 3.py left in tree.
CONFLICT (rename/delete): venv/lib/python3.7/site-packages/astroid/brain/__pycache__/brain_pytest.cpython-37.pyc deleted in version3ascii and renamed to venv/lib/python3.7/site-packages/astroid/brain/brain_pytest 4.py in HEAD. Version HEAD of venv/lib/python3.7/site-packages/astroid/brain/brain_pytest 4.py left in tree.
CONFLICT (rename/delete): venv/lib/python3.7/site-packages/astroid/brain/__pycache__/brain_pytest.cpython-37 2.pyc deleted in version3ascii and renamed to venv/lib/python3.7/site-packages/astroid/brain/brain_pytest 3.py in HEAD. Version HEAD of venv/lib/python3.7/site-packages/astroid/brain/brain_pytest 3.py left in tree.
CONFLICT (rename/delete): venv/lib/python3.7/site-packages/astroid/brain/__pycache__/brain_pkg_resources 2.cpython-37.pyc deleted in version3ascii and renamed to venv/lib/python3.7/site-packages/astroid/brain/brain_pkg_resources 4.py in HEAD. Version HEAD of venv/lib/python3.7/site-packages/astroid/brain/brain_pkg_resources 4.py left in tree.
CONFLICT (rename/delete): venv/lib/python3.7/site-packages/astroid/brain/__pycache__/brain_pkg_resources.cpython-37.pyc deleted in version3ascii and renamed to venv/lib/python3.7/site-packages/astroid/brain/brain_pkg_resources 3.py in HEAD. Version HEAD of venv/lib/python3.7/site-packages/astroid/brain/brain_pkg_resources 3.py left in tree.
CONFLICT (rename/delete): venv/lib/python3.7/site-packages/astroid/brain/__pycache__/brain_numpy_random_mtrand.cpython-37.pyc deleted in version3ascii and renamed to venv/lib/python3.7/site-packages/astroid/brain/brain_numpy_random_mtrand 4.py in HEAD. Version HEAD of venv/lib/python3.7/site-packages/astroid/brain/brain_numpy_random_mtrand 4.py left in tree.
CONFLICT (rename/delete): venv/lib/python3.7/site-packages/astroid/brain/__pycache__/brain_numpy_random_mtrand.cpython-37 2.pyc deleted in version3ascii and renamed to venv/lib/python3.7/site-packages/astroid/brain/brain_numpy_random_mtrand 3.py in HEAD. Version HEAD of venv/lib/python3.7/site-packages/astroid/brain/brain_numpy_random_mtrand 3.py left in tree.
CONFLICT (rename/delete): venv/lib/python3.7/site-packages/astroid/brain/__pycache__/brain_numpy_ndarray.cpython-37.pyc deleted in version3ascii and renamed to venv/lib/python3.7/site-packages/astroid/brain/brain_numpy_ndarray 4.py in HEAD. Version HEAD of venv/lib/python3.7/site-packages/astroid/brain/brain_numpy_ndarray 4.py left in tree.
CONFLICT (rename/delete): venv/lib/python3.7/site-packages/astroid/brain/__pycache__/brain_numpy_ndarray.cpython-37 2.pyc deleted in version3ascii and renamed to venv/lib/python3.7/site-packages/astroid/brain/brain_numpy_ndarray 3.py in HEAD. Version HEAD of venv/lib/python3.7/site-packages/astroid/brain/brain_numpy_ndarray 3.py left in tree.
CONFLICT (rename/delete): venv/lib/python3.7/site-packages/astroid/brain/__pycache__/brain_numpy_core_umath 2.cpython-37.pyc deleted in version3ascii and renamed to venv/lib/python3.7/site-packages/astroid/brain/brain_numpy_core_umath 4.py in HEAD. Version HEAD of venv/lib/python3.7/site-packages/astroid/brain/brain_numpy_core_umath 4.py left in tree.
CONFLICT (rename/delete): venv/lib/python3.7/site-packages/astroid/brain/__pycache__/brain_numpy_core_umath.cpython-37.pyc deleted in version3ascii and renamed to venv/lib/python3.7/site-packages/astroid/brain/brain_numpy_core_umath 3.py in HEAD. Version HEAD of venv/lib/python3.7/site-packages/astroid/brain/brain_numpy_core_umath 3.py left in tree.
CONFLICT (rename/delete): venv/lib/python3.7/site-packages/astroid/brain/__pycache__/brain_numpy_core_numerictypes.cpython-37.pyc deleted in version3ascii and renamed to venv/lib/python3.7/site-packages/astroid/brain/brain_numpy_core_numerictypes 2.py in HEAD. Version HEAD of venv/lib/python3.7/site-packages/astroid/brain/brain_numpy_core_numerictypes 2.py left in tree.
CONFLICT (rename/delete): venv/lib/python3.7/site-packages/astroid/brain/__pycache__/brain_http 2.cpython-37.pyc deleted in version3ascii and renamed to venv/lib/python3.7/site-packages/astroid/brain/brain_http 4.py in HEAD. Version HEAD of venv/lib/python3.7/site-packages/astroid/brain/brain_http 4.py left in tree.
CONFLICT (rename/delete): venv/lib/python3.7/site-packages/astroid/brain/__pycache__/brain_http.cpython-37 2.pyc deleted in version3ascii and renamed to venv/lib/python3.7/site-packages/astroid/brain/brain_http 3.py in HEAD. Version HEAD of venv/lib/python3.7/site-packages/astroid/brain/brain_http 3.py left in tree.
CONFLICT (rename/delete): venv/lib/python3.7/site-packages/astroid/brain/__pycache__/brain_curses.cpython-37.pyc deleted in version3ascii and renamed to venv/lib/python3.7/site-packages/astroid/brain/brain_curses 4.py in HEAD. Version HEAD of venv/lib/python3.7/site-packages/astroid/brain/brain_curses 4.py left in tree.
CONFLICT (rename/delete): venv/lib/python3.7/site-packages/astroid/brain/__pycache__/brain_curses.cpython-37 2.pyc deleted in version3ascii and renamed to venv/lib/python3.7/site-packages/astroid/brain/brain_curses 3.py in HEAD. Version HEAD of venv/lib/python3.7/site-packages/astroid/brain/brain_curses 3.py left in tree.
CONFLICT (rename/delete): venv/lib/python3.7/site-packages/astroid/brain/__pycache__/brain_collections 2.cpython-37.pyc deleted in version3ascii and renamed to venv/lib/python3.7/site-packages/astroid/brain/brain_collections 4.py in HEAD. Version HEAD of venv/lib/python3.7/site-packages/astroid/brain/brain_collections 4.py left in tree.
CONFLICT (rename/delete): venv/lib/python3.7/site-packages/astroid/brain/__pycache__/brain_collections.cpython-37.pyc deleted in version3ascii and renamed to venv/lib/python3.7/site-packages/astroid/brain/brain_collections 3.py in HEAD. Version HEAD of venv/lib/python3.7/site-packages/astroid/brain/brain_collections 3.py left in tree.
Automatic merge failed; fix conflicts and then commit the result.

Then, when I run git mergetool, I get:

git mergetool
Merging:
venv/lib/python3.7/site-packages/astroid/brain/brain_collections 3.py
venv/lib/python3.7/site-packages/astroid/brain/brain_collections 4.py
venv/lib/python3.7/site-packages/astroid/brain/brain_curses 3.py
venv/lib/python3.7/site-packages/astroid/brain/brain_curses 4.py
venv/lib/python3.7/site-packages/astroid/brain/brain_http 3.py
venv/lib/python3.7/site-packages/astroid/brain/brain_http 4.py
venv/lib/python3.7/site-packages/astroid/brain/brain_numpy_core_numerictypes 2.py
venv/lib/python3.7/site-packages/astroid/brain/brain_numpy_core_umath 3.py
venv/lib/python3.7/site-packages/astroid/brain/brain_numpy_core_umath 4.py
venv/lib/python3.7/site-packages/astroid/brain/brain_numpy_ndarray 3.py
venv/lib/python3.7/site-packages/astroid/brain/brain_numpy_ndarray 4.py
venv/lib/python3.7/site-packages/astroid/brain/brain_numpy_random_mtrand 3.py
venv/lib/python3.7/site-packages/astroid/brain/brain_numpy_random_mtrand 4.py
venv/lib/python3.7/site-packages/astroid/brain/brain_pkg_resources 3.py
venv/lib/python3.7/site-packages/astroid/brain/brain_pkg_resources 4.py
venv/lib/python3.7/site-packages/astroid/brain/brain_pytest 3.py
venv/lib/python3.7/site-packages/astroid/brain/brain_pytest 4.py
venv/lib/python3.7/site-packages/astroid/brain/brain_re 3.py
venv/lib/python3.7/site-packages/astroid/brain/brain_re 4.py
venv/lib/python3.7/site-packages/astroid/brain/brain_responses 3.py
venv/lib/python3.7/site-packages/astroid/brain/brain_responses 4.py
venv/lib/python3.7/site-packages/astroid/brain/brain_scipy_signal 3.py
venv/lib/python3.7/site-packages/astroid/brain/brain_scipy_signal 4.py
venv/lib/python3.7/site-packages/astroid/brain/brain_ssl 3.py
venv/lib/python3.7/site-packages/astroid/brain/brain_ssl 4.py
venv/lib/python3.7/site-packages/astroid/brain/brain_subprocess 3.py

Deleted merge conflict for 'venv/lib/python3.7/site-packages/astroid/brain/brain_collections 3.py':
  {local}: created file
  {remote}: deleted
Use (c)reated or (d)eleted file, or (a)bort? a
Continue merging other unresolved paths [y/n]? n

Does anyone know what is going on here?

I don't recall renaming or deleting any venv/lib/* files.

(This is part of a Heroku web app — i.e., the app is currently being run off of master. Not sure if that's relevant.)

python git
1 Answers

First, note that .gitignore content itself never has any direct effect on a merge. That is because git merge merges the contents of commits, which are already committed and cannot be changed. They have the files that they have. No power on Earth, or anywhere else, can change them. Your git merge is merging some existing commits, in preparation for making a new commit.

I have run git rm '*.pyc' in both files ...

Do you mean "in both commits"? "In both files" makes little sense here.

I don't recall renaming or deleting any venv/lib/* files.

If venv/lib contained *.pyc files, and you ran the above git rm, you would remove those *.pyc files from both your work-tree and Git's index. Once the files are out of Git's index, then the existing *.pyc entry in an existing .gitignore can take effect, preventing future *.pyc files from entering Git's index via your work-tree. A subsequent commit would then lack those *.pyc files.

I'll just look at the first conflict here, and split up long lines for posting purposes only:

CONFLICT (rename/delete):
venv/lib/python3.7/site-packages/
astroid/brain/__pycache__/brain_subprocess.cpython-37 2.pyc
deleted in version3ascii and renamed to
venv/lib/python3.7/site-packages/
astroid/brain/brain_subprocess 3.py
in HEAD. ...

All this really means is that:

  • the merge base commit contains a file named .../__pycache__/brain_subprocess.cpython-37 2.pyc;
  • the version3ascii commit lacks this file; and
  • the HEAD version also lacks this file, but has a new file named .../astroid/brain/brain_subprocess 3.py

and the HEAD contents of that new name resemble the merge base contents under the old name, enough so that git merge decided that whoever made the changes between the merge base commit and the HEAD commit must have renamed (and perhaps also modified) the merge-base copy of the file.

It seems much more likely that the merge base commit has these *.pyc files and all these venv/* files, and that the *.pyc files are properly removed in both branch-tip commits (version3ascii branch tip, and current branch tip). However, some venv/* files are present in HEAD, but probably not in version3ascii (otherwise Git probably would have detected a similar rename there). It also seems likely that the .py file isn't a renamed-and-modified copy of the .pyc file and Git's similarity detector has merely mis-detected this as a rename.

There are numerous paths forward. For instance:

  • If no venv/* files should be in either branch-tip commit, you can just make two new branch-tip commits that lack those files. Now Git won't find similar-looking files to claim as renames, that make Git believe things that aren't true.

  • If you don't want to make new commits, you can abort this merge and re-run with an extended-argument (-X argument) that sets the rename threshold higher or turns off the rename detector entirely, e.g., git merge -Xfind-renames=99 to limit it to 99% similar files, instead of 50% similar files.

  • Or, you can simply adjust everything in Git's index manually. The fact is, the merge has stopped with a merge conflict. It is now your job to arrange to have the correct merge results. These need not match any of the three input commits, although it's likely that the correct merge uses all three inputs in some way. Since git merge has come to a complete halt, you now have full control over what will be in its index when you eventually run git merge --continue or git commit to finish the merge. You can run git rm -r . to remove literally everything, construct all new files from whole cloth, and git add.

(Probably one of the other options is more useful than the nuke-and-pave one, and even if you do go for a nuke-and-pave—remove and re-create—you probably don't want to do it wholesale like this.)

user_12564520
Related