Appending encrypted data to file

user_16946838 0
Viewed 73

I am using the cryptography library for python. My goal is to take a string, encrypt it and then write to to a file.

This may be done multiple times, with each appending to the end of the file additional data; which is also encrypted.

I have tried a few solutions, such as:

  1. Using the hazmat level API to avoid as much meta data stored in the encrypted text.
  2. Writing each encrypted string to a new line in a text file.

This is the code that uses ECB mode and the hazmat API. It attempts to read the file and decrypt line by line. I understand it is unsafe, my main use is to log this data only locally to a file and then use a safe PKCS over the wire.

from cryptography import fernet

key = 'WqSAOfEoOdSP0c6i1CiyoOpTH2Gma3ff_G3BpDx52sE='
crypt_obj = fernet.Fernet(key)
file_handle = open('test.txt', 'a')

data = 'Hello1'
data = crypt_obj.encrypt(data.encode())
file_handle.write(data.decode() + '\n')
file_handle.close()


file_handle_two = open('test.txt', 'a')
data_two = 'Hello2'
data_two = crypt_obj.encrypt(data_two.encode())
file_handle_two.write(data_two.decode() + '\n')
file_handle_two.close()


file_read = open('test.txt', 'r')
file_lines = file_read.readlines()
file_content = ''
for line in file_lines:
    line = line[:-2]
    file_content = crypt_obj.decrypt(line.encode()).decode()
    print(file_content)
file_read.close()

For the code above I get the following error:

Traceback (most recent call last):
  File "C:\Dev\Python\local_crypt_test\venv\lib\site-packages\cryptography\fernet.py", line 110, in _get_unverified_token_data
    data = base64.urlsafe_b64decode(token)
  File "C:\Users\19097\AppData\Local\Programs\Python\Python39\lib\base64.py", line 133, in urlsafe_b64decode
    return b64decode(s)
  File "C:\Users\19097\AppData\Local\Programs\Python\Python39\lib\base64.py", line 87, in b64decode
    return binascii.a2b_base64(s)
binascii.Error: Incorrect padding

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Dev\Python\local_crypt_test\main.py", line 25, in <module>
    file_content = crypt_obj.decrypt(line.encode()).decode()
  File "C:\Dev\Python\local_crypt_test\venv\lib\site-packages\cryptography\fernet.py", line 83, in decrypt
    timestamp, data = Fernet._get_unverified_token_data(token)
  File "C:\Dev\Python\local_crypt_test\venv\lib\site-packages\cryptography\fernet.py", line 112, in _get_unverified_token_data
    raise InvalidToken
cryptography.fernet.InvalidToken

Process finished with exit code 1

These examples are only to demonstrate the issue, my real code looks much different so you may ignore errors in the example that do not pertain to my main issue. That is, appending encrypted data to a file and decrypting/reading that data from the file at a later time. The file does not need to be in any specific format, as long as it can be read from and decrypted to obtain the original message. Also, the mode of operation is not tied to ECB, if your example uses another type, that works too.

I am honestly stumped and would appreciate any help!

aes cryptography encryption file python
2 Answers

There are a couple details at play here...

1. Trailing newline character(s) are included in each line

When you loop through file_lines, each line includes the trailing newline character(s).

I say "character(s)" because this can vary based on the platform (e.g. Linux/macOS = '\n' versus Windows = '\r\n').

2. base64 decoding silently discards invalid characters

Fernet.encrypt(data) returns a bytes instance containing a base64 encoded "Fernet token".

Conversely, the first step Fernet.decrypt(token) takes is decoding the token by calling base64.urlsafe_b64decode(). This function uses the default non-validating behavior in which characters not within the base64 set are discarded (described here).

Note: This is why the answer from TheTS happens to work despite leaving the extraneous newline character intact.

Solution

I'd recommend making sure you provide Fernet.decrypt() the token exactly as produced by Fernet.encrypt(). I'm guessing this is what you were trying to do by stripping the last two characters.

Here's an approach that should be safe and not platform dependent.

Here's a code example that demonstrates this:

#!/usr/bin/python3

from cryptography import fernet

to_encrypt = ['Hello1', 'Hello2']
output_file = 'test.txt'

key = 'WqSAOfEoOdSP0c6i1CiyoOpTH2Gma3ff_G3BpDx52sE='
crypt = fernet.Fernet(key)

print("ENCRYPTING...")
for data in to_encrypt:
    data_bytes = data.encode('utf-8')
    token_bytes = crypt.encrypt(data_bytes)
    print(f'data: {data}')
    print(f'token_bytes: {token_bytes}\n')
    with open(output_file, 'a', newline='\n') as f:
        f.write(token_bytes.decode('utf-8') + '\n')

print("\nDECRYPTING...")
with open(output_file, 'r') as f:
    for line in f:
        # Create a copy of line which shows the trailing newline.
        line_escaped = line.encode('unicode_escape').decode('utf-8')
        line_stripped = line.rstrip('\n')
        token_bytes = line_stripped.encode('utf-8')
        data = crypt.decrypt(token_bytes).decode('utf-8')
        print(f'line_escaped: {line_escaped}')
        print(f'token_bytes: {token_bytes}')
        print(f'decrypted data: {data}\n')

Output:

Note the trailing newline when line escaped is printed.

$ python3 solution.py
ENCRYPTING...
data: Hello1
token_bytes: b'gAAAAABi-LAo-h8w-ayc267hrLbswMZtkT4RQQ9wt0EusYNrZGjuzbpyRLoKDZZF4oQPOU-iH1PnCc7vSIOoTVMLlCFnHTkN6A=='

data: Hello2
token_bytes: b'gAAAAABi-LAoHUT8Iu1bVMcGSIrFRvtVZQFh4O52XYSCgd0leYWS-n38irhv3Ch7oEx6SXazHwAL7a57ncFoMJTQQAms52yf3w=='


DECRYPTING...
line_escaped: gAAAAABi-LAo-h8w-ayc267hrLbswMZtkT4RQQ9wt0EusYNrZGjuzbpyRLoKDZZF4oQPOU-iH1PnCc7vSIOoTVMLlCFnHTkN6A==\n
token_bytes: b'gAAAAABi-LAo-h8w-ayc267hrLbswMZtkT4RQQ9wt0EusYNrZGjuzbpyRLoKDZZF4oQPOU-iH1PnCc7vSIOoTVMLlCFnHTkN6A=='
decrypted data: Hello1

line_escaped: gAAAAABi-LAoHUT8Iu1bVMcGSIrFRvtVZQFh4O52XYSCgd0leYWS-n38irhv3Ch7oEx6SXazHwAL7a57ncFoMJTQQAms52yf3w==\n
token_bytes: b'gAAAAABi-LAoHUT8Iu1bVMcGSIrFRvtVZQFh4O52XYSCgd0leYWS-n38irhv3Ch7oEx6SXazHwAL7a57ncFoMJTQQAms52yf3w=='
decrypted data: Hello2
user_3738150 
from cryptography import fernet

key = 'WqSAOfEoOdSP0c6i1CiyoOpTH2Gma3ff_G3BpDx52sE='
crypt_obj = fernet.Fernet(key)
file_handle = open('test.txt', 'a')

data = 'Hello1'
data = crypt_obj.encrypt(data.encode('utf-8'))
file_handle.write(data.decode('utf-8') + '\n')
file_handle.close()


file_handle_two = open('test.txt', 'a')
data_two = 'Hello2'
data_two = crypt_obj.encrypt(data_two.encode('utf-8'))
file_handle_two.write(data_two.decode('utf-8') + '\n')
file_handle_two.close()


file_read = open('test.txt', 'r')
file_lines = file_read.readlines()
file_content = ''
for line in file_lines:
    # line = line[:-2]
    file_content = crypt_obj.decrypt(line.encode('utf-8')).decode()
    print(file_content)
file_read.close()

By removing the last characters from the string you also remove important characters for decoding.

user_197294830
Related