Why can't Python's raw string literals end with a single backslash?

Viewed 106296

Technically, any odd number of backslashes, as described in the documentation.

>>> r'\'
  File "<stdin>", line 1
    r'\'
       ^
SyntaxError: EOL while scanning string literal
>>> r'\\'
'\\\\'
>>> r'\\\'
  File "<stdin>", line 1
    r'\\\'
         ^
SyntaxError: EOL while scanning string literal

It seems like the parser could just treat backslashes in raw strings as regular characters (isn't that what raw strings are all about?), but I'm probably missing something obvious.

13 Answers

The reason is explained in the part of that section which I highlighted in bold:

String quotes can be escaped with a backslash, but the backslash remains in the string; for example, r"\"" is a valid string literal consisting of two characters: a backslash and a double quote; r"\" is not a valid string literal (even a raw string cannot end in an odd number of backslashes). Specifically, a raw string cannot end in a single backslash (since the backslash would escape the following quote character). Note also that a single backslash followed by a newline is interpreted as those two characters as part of the string, not as a line continuation.

So raw strings are not 100% raw, there is still some rudimentary backslash-processing.

That's the way it is! I see it as one of those small defects in python!

I don't think there's a good reason for it, but it's definitely not parsing; it's really easy to parse raw strings with \ as a last character.

The catch is, if you allow \ to be the last character in a raw string then you won't be able to put " inside a raw string. It seems python went with allowing " instead of allowing \ as the last character.

However, this shouldn't cause any trouble.

If you're worried about not being able to easily write windows folder pathes such as c:\mypath\ then worry not, for, you can represent them as r"C:\mypath", and, if you need to append a subdirectory name, don't do it with string concatenation, for it's not the right way to do it anyway! use os.path.join

>>> import os
>>> os.path.join(r"C:\mypath", "subfolder")
'C:\\mypath\\subfolder'

Since \" is allowed inside the raw string. Then it can't be used to identify the end of the string literal.

Why not stop parsing the string literal when you encounter the first "?

If that was the case, then \" wouldn't be allowed inside the string literal. But it is.

The reason for why r'\' is syntactical incorrect is that although the string expression is raw the used quotes (single or double) always have to be escape since they would mark the end of the quote otherwise. So if you want to express a single quote inside single quoted string, there is no other way than using \'. Same applies for double quotes.

But you could use:

'\\'

Another user who has since deleted their answer (not sure if they'd like to be credited) suggested that the Python language designers may be able to simplify the parser design by using the same parsing rules and expanding escaped characters to raw form as an afterthought (if the literal was marked as raw).

I thought it was an interesting idea and am including it as community wiki for posterity.

Naive raw strings

The naive idea of a raw string is

If I put an r in front of a pair of quotes, I can put whatever I want between the quotes and it will mean itself.

Unfortunately, this does not work, because if the whatever happens to contain a quote, the raw string would end at that point.

It is simply impossible that I can put "whatever I want" between fixed delimiters, because some of it could look like the terminating delimiter -- no matter what that delimiter is.

Real-world raw strings (variant 1)

One possible approach to this problem would be to say

If I put an r in front of a pair of quotes, I can put whatever I want between the quotes as long as it does not contain a quote and it will mean itself.

This restriction sounds harsh, until one recognizes that Python's large offering of quotes can accommodate most situations with this rule. The following are all valid Python quotes:

'
"
'''
"""

With this many possibilities for the delimiter, almost anything can be made to work. About the only exception would be if the string literal is supposed to contain a complete list of all allowed Python quotes.

Real-world raw strings (variant 2, as in Python)

Python, however, takes a different route using an extended version of the above rule. It effectively states

If I put an r in front of a pair of quotes, I can put whatever I want between the quotes as long as it does not contain a quote and it will mean itself. If I insist on including a quote, even that is allowed, but I have to put a backslash before it.

So the Python approach is, in a sense, even more liberal than variant 1 above -- but it has the side effect of "mis"interpreting the closing quote as part of the string if the last intended character of the string is a backslash.

Variant 2 is not helpful:

  • If I want the quote in my string, but not the backslash, the allowed version of my string literal will not be what I need.
    However, given the three different other kinds of quotes I have at my disposal, I will probably just pick one of those and my problem will be solved -- so this is not problematic case.
  • The problematic case is this one: If I want my string to end with a backslash, I am at a loss. I need to resort to concatenating a non-raw string literal containing the backslash.

Conclusion

After writing this, I go with several of the other posters that variant 1 would have been easier to understand and to accept and therefore more pythonic. That's life!

Comming from C it pretty clear to me that a single \ works as escape character allowing you to put special characters such as newlines, tabs and quotes into strings.

That does indeed disallow \ as last character since it will escape the " and make the parser choke. But as pointed out earlier \ is legal.

some tips :

1) if you need to manipulate backslash for path then standard python module os.path is your friend. for example :

os.path.normpath('c:/folder1/')

2) if you want to build strings with backslash in it BUT without backslash at the END of your string then raw string is your friend (use 'r' prefix before your literal string). for example :

r'\one \two \three'

3) if you need to prefix a string in a variable X with a backslash then you can do this :

X='dummy'
bs=r'\ ' # don't forget the space after backslash or you will get EOL error
X2=bs[0]+X  # X2 now contains \dummy

4) if you need to create a string with a backslash at the end then combine tip 2 and 3 :

voice_name='upper'
lilypond_display=r'\DisplayLilyMusic \ ' # don't forget the space at the end
lilypond_statement=lilypond_display[:-1]+voice_name

now lilypond_statement contains "\DisplayLilyMusic \upper"

long live python ! :)

n3on

I encountered this problem and found a partial solution which is good for some cases. Despite python not being able to end a string with a single backslash, it can be serialized and saved in a text file with a single backslash at the end. Therefore if what you need is saving a text with a single backslash on you computer, it is possible:

x = 'a string\\' 
x
'a string\\' 

# Now save it in a text file and it will appear with a single backslash:

with open("my_file.txt", 'w') as h:
    h.write(x)

BTW it is not working with json if you dump it using python's json library.

Finally, I work with Spyder, and I noticed that if I open the variable in spider's text editor by double clicking on its name in the variable explorer, it is presented with a single backslash and can be copied to the clipboard that way (it's not very helpful for most needs but maybe for some..).

Related