I stumbled upon "UnicodeDecodeError:" after running "python -m pipreqs.pipreqs . " on VSC terminal

Jamiu Shaibu 0
Viewed 35

I wanted to generate requirements.txt of only used packages in a given project. pip freeze works fine but saves all packages in the environment including those that you don't use in your current project, and once you have a bunch of installed packages, it become tedious deleting manually the packages that aren't required for the current project.

I tried pipreqs which is supposed to generate requirements.txt based on used packages in the current project, by running the below approaches in terminal but they throw the same error:

$ python -m  pipreqs.pipreqs .

$ pipreqs

$ pipreqs /project/location

Error:

Traceback (most recent call last):
  File "C:\Program Files\Python39\lib\runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Program Files\Python39\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "C:\Users\myProjects\dataScience\sp_app\advance_app\venv\Scripts\pipreqs.exe\__main__.py", line 7, in <module>
  File "C:\Users\myProjects\dataScience\sp_app\advance_app\venv\lib\site-packages\pipreqs\pipreqs.py", line 488, in main
    init(args)
  File "C:\Users\myProjects\dataScience\sp_app\advance_app\venv\lib\site-packages\pipreqs\pipreqs.py", line 415, in init
    candidates = get_all_imports(input_path,
  File "C:\Users\myProjects\dataScience\sp_app\advance_app\venv\lib\site-packages\pipreqs\pipreqs.py", line 115, in get_all_imports
    contents = f.read()
  File "C:\Program Files\Python39\lib\encodings\cp1252.py", line 23, in decode
    return codecs.charmap_decode(input,self.errors,decoding_table)[0]
UnicodeDecodeError: 'charmap' codec can't decode byte 0x8d in position 1237: character maps to <undefined>

Could anyone help spot the error?

dependencies python-import requirements.txt python
2 Answers

At the point where the crash is happening, pipreqs is thinking that your input is cp1252 but the data contains a character that is not supported within that character set/encoding (eg, byte 0x8d).

You could try passing --encoding argument and experiment with different values (like utf-8 or utf-16)

rasjani0

Even though @rasjani spotted a significant point of the cause of the error, when I implemented the rule, a new error popped up again and took me quite some couple of minutes to reflex. Here was what happened.

$ pipreqs --encoding=utf8 C:\Users\myProjects\dataScience\sp_app\advance_app

New Error:

Traceback (most recent call last):
  File "C:\Program Files\Python39\lib\runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Program Files\Python39\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "C:\Users\myProjects\dataScience\sp_app\advance_app\venv\Scripts\pipreqs.exe\__main__.py", line 7, in <module>
  File "C:\Users\myProjects\dataScience\sp_app\advance_app\venv\lib\site-packages\pipreqs\pipreqs.py", line 488, in main
    init(args)
  File "C:\Users\myProjects\dataScience\sp_app\advance_app\venv\lib\site-packages\pipreqs\pipreqs.py", line 478, in init
    generate_requirements_file(path, imports, symbol)
  File "C:\Users\myProjects\dataScience\sp_app\advance_app\venv\lib\site-packages\pipreqs\pipreqs.py", line 157, in generate_requirements_file
    with _open(path, "w") as out_file:
  File "C:\Program Files\Python39\lib\contextlib.py", line 117, in __enter__
    return next(self.gen)
  File "C:\Users\myProjects\dataScience\sp_app\advance_app\venv\lib\site-packages\pipreqs\pipreqs.py", line 81, 
in _open
    file = open(filename, mode)
FileNotFoundError: [Errno 2] No such file or directory: 'C:UsersmyProjectsdataSciencesp_appadvance_appsrcpackagesadvance_app\\requirements.txt'

Note: The reason why this happen is because I used single \ for the path instead of \\.
Solution:

$ pipreqs --encoding=utf8 C:\\Users\\myProjects\\dataScience\\sp_app\\advance_app

Output:

INFO: Successfully saved requirements file in C:\Users\myProjects\dataScience\sp_app\advance_app\requirements.txt
Jamiu Shaibu0
Related