Python 3.7.4: 're.error: bad escape \s at position 0'

Viewed 35731

My program looks something like this:

import re
# Escape the string, in case it happens to have re metacharacters
my_str = "The quick brown fox jumped"
escaped_str = re.escape(my_str)
# "The\\ quick\\ brown\\ fox\\ jumped"
# Replace escaped space patterns with a generic white space pattern
spaced_pattern = re.sub(r"\\\s+", r"\s+", escaped_str)
# Raises error

The error is this:

Traceback (most recent call last):
  File "<input>", line 1, in <module>
  File "/home/swfarnsworth/programs/pycharm-2019.2/helpers/pydev/_pydev_bundle/pydev_umd.py", line 197, in runfile
    pydev_imports.execfile(filename, global_vars, local_vars)  # execute the script
  File "/home/swfarnsworth/programs/pycharm-2019.2/helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile
    exec(compile(contents+"\n", file, 'exec'), glob, loc)
  File "/home/swfarnsworth/projects/medaCy/medacy/tools/converters/con_to_brat.py", line 255, in <module>
    content = convert_con_to_brat(full_file_path)
  File "/home/swfarnsworth/projects/my_file.py", line 191, in convert_con_to_brat
    start_ind = get_absolute_index(text_lines, d["start_ind"], d["data_item"])
  File "/home/swfarnsworth/projects/my_file.py", line 122, in get_absolute_index
    entity_pattern_spaced = re.sub(r"\\\s+", r"\s+", entity_pattern_escaped)
  File "/usr/local/lib/python3.7/re.py", line 192, in sub
    return _compile(pattern, flags).sub(repl, string, count)
  File "/usr/local/lib/python3.7/re.py", line 309, in _subx
    template = _compile_repl(template, pattern)
  File "/usr/local/lib/python3.7/re.py", line 300, in _compile_repl
    return sre_parse.parse_template(repl, pattern)
  File "/usr/local/lib/python3.7/sre_parse.py", line 1024, in parse_template
    raise s.error('bad escape %s' % this, len(this))
re.error: bad escape \s at position 0

I get this error even if I remove the two backslashes before the '\s+' or if I make the raw string (r"\\\s+") into a regular string. I checked the Python 3.7 documentation, and it appears that \s is still the escape sequence for white space.

7 Answers

Just try import regex as re instead of import re.

Try fiddling with the backslashes to avoid that regex tries to interpret \s:

spaced_pattern = re.sub(r"\\\s+", "\\\s+", escaped_str)

now

>>> spaced_pattern
'The\\s+quick\\s+brown\\s+fox\\s+jumped'
>>> print(spaced_pattern)
The\s+quick\s+brown\s+fox\s+jumped

But why?

It seems that python tries to interpret \s like it would interpret r"\n" instead of leaving it alone like Python normally does. If you do. For example:

re.sub(r"\\\s+", r"\n+", escaped_str)

yields:

The
+quick
+brown
+fox
+jumped

even if \n was used in a raw string.

The change was introduced in Issue #27030: Unknown escapes consisting of '\' and ASCII letter in regular expressions now are errors.

The code that does the replacement is in sre_parse.py (python 3.7):

        else:
            try:
                this = chr(ESCAPES[this][1])
            except KeyError:
                if c in ASCIILETTERS:
                    raise s.error('bad escape %s' % this, len(this))

This code looks for what's behind a literal \ and tries to replace it by the proper non-ascii character. Obviously s is not in ESCAPES dictionary so the KeyError exception is triggered, then the message you're getting.

On previous versions it just issued a warning:

import warnings
warnings.warn('bad escape %s' % this,
              DeprecationWarning, stacklevel=4)

Looks that we're not alone to suffer from 3.6 to 3.7 upgrade: https://github.com/gi0baro/weppy/issues/227

Here is my simple code, which uses python-binance library and pandas, and it works in one venv with python 3.7, but when i had created new one for another project (python 3.7 as well) it threw the same errors with regex:

import pandas as pd
from binance import Client

api_key = ''
api_secret = ''

client = Client(api_key, api_secret)

timeframe = '1h'
coin = 'ETHUSDT'


def GetOHLC(coin, timeframe):
    frame = pd.DataFrame(client.get_historical_klines(coin, timeframe, '01.01.2015'))
    frame = frame.loc[:, :5]
    frame.columns = ['date', 'open', 'high', 'low', 'close', 'volume']
    frame.set_index('date', inplace=True)        
    frame.to_csv(path_or_buf=(coin+timeframe))


GetOHLC(coin, timeframe)

I had made some research but didn't find suitable solution. Then i compared version of regex lib of workable instance and new one: old one was from 2021 and new one was from 2022. Then i uninstall version of 2022 and install 2021 and it has started to work without any exceptions. Hope it will help in some particular cases.

I guess you might be trying to do:

import re
# Escape the string, in case it happens to have re metacharacters
my_str = "The\\ quick\\ brown\\ fox\\ jumped"
escaped_str = re.escape(my_str)
# "The\\ quick\\ brown\\ fox\\ jumped"
# Replace escaped space patterns with a generic white space pattern
print(re.sub(r"\\\\\\\s+", " ", escaped_str))

Output 1

The quick brown fox jumped

If you might want to have literal \s+, then try this answer or maybe:

import re
# Escape the string, in case it happens to have re metacharacters
my_str = "The\\ quick\\ brown\\ fox\\ jumped"
escaped_str = re.escape(my_str)
print(re.sub(r"\\\\\\\s+", re.escape(r"\s") + '+', escaped_str))

Output 2

The\s+quick\s+brown\s+fox\s+jumped

Or maybe:

import re
# Escape the string, in case it happens to have re metacharacters
my_str = "The\\ quick\\ brown\\ fox\\ jumped"
print(re.sub(r"\s+", "s+", my_str))

Output 3

The\s+quick\s+brown\s+fox\s+jumped

If you wish to simplify/modify/explore the expression, it's been explained on the top right panel of regex101.com. If you'd like, you can also watch in this link, how it would match against some sample inputs.


RegEx Circuit

jex.im visualizes regular expressions:

enter image description here

Demo

Regex engines behave the same way (mostly) when it comes to replacement strings
that are handed to them.
They try to insert the control code equivalent of escaped characters, like tabs crlf's, etc ...
Any escape sequence it doesn't recognize, it just strips off the escape.

Given
spaced_pattern = re.sub(r"\\\s+", r"\s+", escaped_str)

the r"\s+" hands the engine this replacement string \s+.
Since there is no such escape sequence, it just strips off the escape
and inserts s+ into the replace position.

You can see it here https://regex101.com/r/42QCvi/1
There is no error thrown, but it should be since your not getting what you think you should.

In reality, a literal escape should always be escaped
as can be seen here https://regex101.com/r/bzQgfN/1

Nothing new, they just say its an error, but its really a notification warning
that you're not getting what you think.
Been this way for years and years. Sometimes its an error, sometimes not.

In case you are trying to replace anything by a single backslash, both the re and regex packages of Python 3.8.5 cannot do it alone.

The solution I rely on is to split the task between re.sub and Python's replace:

import re
re.sub(r'([0-9.]+)\*([0-9.]+)',r'\1 XBACKSLASHXcdot \2'," 4*2").replace('XBACKSLASHX','\\')
pip uninstall regex

pip install regex==2022.3.2
Related