I'm trying to use Black to format my Django code. It works fine in most directories. However, I get the Unicode decode error UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 0: invalid start byte on the .py files in one directory in particular. The files are in UTF-8, but I have tried multiple different other encodings and have tried deleting and recreating files which hasn't resolved the issue. I'm working in an Anaconda environment, but deactivating that doesn't make any difference. I've also disabled most of my VS Code extensions.
I have looked at many SO postings on this, but none of the suggestions thus far have helped. I'm using VS Code on a Mac and have a feeling it is something to do with the fact that in the past I have edited the same code on a Windows machine using VS Code with WSL (code synced with a GitHub repo), but can't figure out how to address it. I've put a sample Traceback call below. Any help would be much appreciated!
Edit: If I cut and paste any of the files into another directory, then Black works fine and I don't get any encoding error messages. Which suggests there may be a directory-level issue at play?
(base) ➜ healthtic git:(master) ✗ black pages/views.py
Traceback (most recent call last):
File "/Users/jh/opt/anaconda3/bin/black", line 8, in <module>
sys.exit(patched_main())
File "/Users/jh/opt/anaconda3/lib/python3.8/site-packages/black/__init__.py", line 1130, in patched_main
main()
File "/Users/jh/opt/anaconda3/lib/python3.8/site-packages/click/core.py", line 829, in __call__
return self.main(*args, **kwargs)
File "/Users/jh/opt/anaconda3/lib/python3.8/site-packages/click/core.py", line 782, in main
rv = self.invoke(ctx)
File "/Users/jh/opt/anaconda3/lib/python3.8/site-packages/click/core.py", line 1066, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/Users/jh/opt/anaconda3/lib/python3.8/site-packages/click/core.py", line 610, in invoke
return callback(*args, **kwargs)
File "/Users/jh/opt/anaconda3/lib/python3.8/site-packages/click/decorators.py", line 21, in new_func
return f(get_current_context(), *args, **kwargs)
File "/Users/jh/opt/anaconda3/lib/python3.8/site-packages/black/__init__.py", line 411, in main
sources = get_sources(
File "/Users/jh/opt/anaconda3/lib/python3.8/site-packages/black/__init__.py", line 477, in get_sources
gitignore = get_gitignore(root)
File "/Users/jh/opt/anaconda3/lib/python3.8/site-packages/black/files.py", line 123, in get_gitignore
lines = gf.readlines()
File "/Users/jh/opt/anaconda3/lib/python3.8/codecs.py", line 322, in decode
(result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 0: invalid start byte
Further edit: As mentioned above if I move files out of this directory, Black works fine, if I move them back (without any changes), the 0xFF unicode error recurs. I have attached my directory structure below, as it may be relevant that I have webpack and node modules in there for some of the frontend. My impression is that maybe a BOM may be being added somehow in the directory?
├── Dockerfile
├── accounts
│ ├── __init__.py
│ ├── __pycache__
│ ├── admin.py
│ ├── apps.py
│ ├── forms.py
│ ├── migrations
│ ├── models.py
│ ├── tests.py
│ ├── urls.py
│ └── views.py
├── appointments
│ ├── __init__.py
│ ├── __pycache__
│ ├── admin.py
│ ├── apps.py
│ ├── forms.py
│ ├── migrations
│ ├── models.py
│ ├── tests.py
│ ├── urls.py
│ └── views.py
├── cloud_sql_proxy
├── cloudmigrate.yaml
├── config
│ ├── __init__.py
│ ├── __pycache__
│ ├── asgi.py
│ ├── basesettings.py
│ ├── settings.py
│ ├── urls.py
│ └── wsgi.py
├── db.sqlite3
├── docker-compose.yaml
├── front-end
│ ├── App\ copy.js
│ ├── App.js
│ ├── index\ copy.js
│ └── index.js
├── logfile
├── manage.py
├── node_modules
│ ├── @babel
│ ├── @discoveryjs
│ ├── @types
│ ├── @webassemblyjs
│ ├── @webpack-cli
│ ├── @xtuc
│ ├── acorn
│ ├── ajv
│ ├── ajv-keywords
│ ├── ansi-styles
│ ├── babel-loader
│ ├── babel-plugin-dynamic-import-node
│ ├── babel-plugin-polyfill-corejs2
│ ├── babel-plugin-polyfill-corejs3
│ ├── babel-plugin-polyfill-regenerator
│ ├── big.js
│ ├── browserslist
│ ├── buffer-from
│ ├── call-bind
│ ├── caniuse-lite
│ ├── chalk
│ ├── chrome-trace-event
│ ├── clone-deep
│ ├── color-convert
│ ├── color-name
│ ├── colorette
│ ├── commander
│ ├── commondir
│ ├── convert-source-map
│ ├── core-js-compat
│ ├── cross-spawn
│ ├── debug
│ ├── define-properties
│ ├── electron-to-chromium
│ ├── emojis-list
│ ├── enhanced-resolve
│ ├── envinfo
│ ├── es-module-lexer
│ ├── escalade
│ ├── escape-string-regexp
│ ├── eslint-scope
│ ├── esrecurse
│ ├── estraverse
│ ├── esutils
│ ├── events
│ ├── execa
│ ├── fast-deep-equal
│ ├── fast-json-stable-stringify
│ ├── fastest-levenshtein
│ ├── find-cache-dir
│ ├── find-up
│ ├── function-bind
│ ├── gensync
│ ├── get-intrinsic
│ ├── get-stream
│ ├── glob-to-regexp
│ ├── globals
│ ├── graceful-fs
│ ├── has
│ ├── has-flag
│ ├── has-symbols
│ ├── human-signals
│ ├── import-local
│ ├── interpret
│ ├── is-core-module
│ ├── is-plain-object
│ ├── is-stream
│ ├── isexe
│ ├── isobject
│ ├── jest-worker
│ ├── js-tokens
│ ├── jsesc
│ ├── json-parse-better-errors
│ ├── json-schema-traverse
│ ├── json5
│ ├── kind-of
│ ├── loader-runner
│ ├── loader-utils
│ ├── locate-path
│ ├── lodash
│ ├── lodash.debounce
│ ├── loose-envify
│ ├── make-dir
│ ├── merge-stream
│ ├── mime-db
│ ├── mime-types
│ ├── mimic-fn
│ ├── minimist
│ ├── ms
│ ├── neo-async
│ ├── node-releases
│ ├── npm-run-path
│ ├── object-assign
│ ├── object-keys
│ ├── object.assign
│ ├── onetime
│ ├── p-limit
│ ├── p-locate
│ ├── p-try
│ ├── path-exists
│ ├── path-key
│ ├── path-parse
│ ├── pkg-dir
│ ├── punycode
│ ├── randombytes
│ ├── react
│ ├── react-dom
│ ├── rechoir
│ ├── regenerate
│ ├── regenerate-unicode-properties
│ ├── regenerator-runtime
│ ├── regenerator-transform
│ ├── regexpu-core
│ ├── regjsgen
│ ├── regjsparser
│ ├── resolve
│ ├── resolve-cwd
│ ├── resolve-from
│ ├── safe-buffer
│ ├── scheduler
│ ├── schema-utils
│ ├── semver
│ ├── serialize-javascript
│ ├── shallow-clone
│ ├── shebang-command
│ ├── shebang-regex
│ ├── signal-exit
│ ├── source-list-map
│ ├── source-map
│ ├── source-map-support
│ ├── strip-final-newline
│ ├── supports-color
│ ├── tapable
│ ├── terser
│ ├── terser-webpack-plugin
│ ├── to-fast-properties
│ ├── unicode-canonical-property-names-ecmascript
│ ├── unicode-match-property-ecmascript
│ ├── unicode-match-property-value-ecmascript
│ ├── unicode-property-aliases-ecmascript
│ ├── uri-js
│ ├── v8-compile-cache
│ ├── watchpack
│ ├── webpack
│ ├── webpack-cli
│ ├── webpack-merge
│ ├── webpack-sources
│ ├── which
│ ├── wildcard
│ └── yocto-queue
├── notes.md
├── package-lock.json
├── package.json
├── pages
│ ├── __init__.py
│ ├── __pycache__
│ ├── admin.py
│ ├── apps.py
│ ├── migrations
│ ├── models.py
│ ├── tests.py
│ ├── urls.py
│ └── views.py
├── patient
│ ├── __init__.py
│ ├── __pycache__
│ ├── admin.py
│ ├── apps.py
│ ├── migrations
│ ├── models.py
│ ├── serializers.py
│ ├── tests.py
│ ├── urls.py
│ └── views.py
├── patientinfo
│ ├── __init__.py
│ ├── __pycache__
│ ├── admin.py
│ ├── apps.py
│ ├── migrations
│ ├── models.py
│ ├── tests.py
│ ├── urls.py
│ └── views.py
├── readme.md
├── requirements.txt
├── static
│ ├── css
│ ├── images
│ ├── index-bundle.js
│ ├── index-bundle.js.LICENSE.txt
│ └── js-bundle.js
├── staticfiles
│ ├── admin
│ ├── css
│ └── images
├── templates
│ ├── _base.html
│ ├── account
│ ├── appointments
│ ├── frontend.html
│ ├── hello_webpack.html
│ ├── home.html
│ ├── patient
│ └── patientinfo
├── todos.md
└── webpack.config.js