Open
Description
I just ran codespell on all python peps with chardet -e. codespell crashed:
$ codespell --write-changes -i3 -C 5 -H -f -e --count -s --builtin clear,rare,names
Traceback (most recent call last):
File "/Users/corneliusromer/.local/bin/codespell", line 8, in <module>
sys.exit(_script_main())
^^^^^^^^^^^^^^
File "/Users/corneliusromer/.local/pipx/venvs/codespell/lib/python3.12/site-packages/codespell_lib/_codespell.py", line 1121, in _script_main
return main(*sys.argv[1:])
^^^^^^^^^^^^^^^^^^^
File "/Users/corneliusromer/.local/pipx/venvs/codespell/lib/python3.12/site-packages/codespell_lib/_codespell.py", line 1305, in main
bad_count += parse_file(
^^^^^^^^^^^
File "/Users/corneliusromer/.local/pipx/venvs/codespell/lib/python3.12/site-packages/codespell_lib/_codespell.py", line 963, in parse_file
lines, encoding = file_opener.open(filename)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/corneliusromer/.local/pipx/venvs/codespell/lib/python3.12/site-packages/codespell_lib/_codespell.py", line 232, in open
return self.open_with_chardet(filename)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/corneliusromer/.local/pipx/venvs/codespell/lib/python3.12/site-packages/codespell_lib/_codespell.py", line 257, in open_with_chardet
lines = f.readlines()
^^^^^^^^^^^^^
File "/opt/homebrew/Cellar/[email protected]/3.12.5/Frameworks/Python.framework/Versions/3.12/lib/python3.12/encodings/cp1254.py", line 23, in decode
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
UnicodeDecodeError: 'charmap' codec can't decode byte 0x90 in position 1349: character maps to <undefined>The chardet error should be caught and rethrown with a better error message for the end user - I don't even know which file it is based on the error message as is!