Django - (1366, "Incorrect string value:... error

user_8459497 0
Viewed 4881

I am getting the following error when I try to add a new record via django admin:

OperationalError at /admin/competition/sport/add/ (1366, "Incorrect string value: '\xC4\x9F\xC3\xBC' for column 'object_repr' at row 1") Request Method: POST Request URL: http://127.0.0.1:8000/admin/competition/sport/add/ Django Version: 1.11.4 Exception Type: OperationalError Exception Value:
(1366, "Incorrect string value: '\xC4\x9F\xC3\xBC' for column 'object_repr' at row 1")

Here is the model:

class Sport(models.Model):
    is_team = models.BooleanField(_("Is Team"))
    name = models.CharField(_("Name"), max_length=200)

And options for mysql backend at my settings.py:

'OPTIONS': {
        'charset': 'utf8mb4',
        'init_command': 'SET character_set_connection=utf8mb4;'
                        'SET collation_connection=utf8mb4_unicode_ci;'
                        "SET NAMES 'utf8mb4';"
                        "SET CHARACTER SET utf8mb4;"
    },

I have updated all of the tables and columns to use utf8mb4 as described here

Nothing has worked so far. I get no error when I try to insert a record that contains unicode characters using mysql shell, but the django admin gives the error above.

Edit:

Interestingly the code below works perfectly when i try to insert a record manually:

Sport(is_team=True, name="ÜĞüiğÇÖ").save()

EDIT 2:

I finally figured what causes the problem. Django admin logs every action by default (including the data, in my case it was some weird unicode text). It turns out 'django_admin_log' table has 'latin1' charset, so i simply converted it into utf8:

ALTER TABLE django_admin_log CONVERT TO CHARACTER SET utf8 COLLATE utf8_unicode_ci;

Then everything worked fine.

mysql django python
4 Answers

You have to modify the django_admin_log character set

ALTER TABLE django_admin_log CHANGE object_repr object_repr VARCHAR(191) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
user_12316820 
class Employee(models.Model):
        name = models.CharField(max_length=100)

        def save(self, *args, **kwargs):
           super(Employee, self).save(*args, **kwargs)
           self.name = str(self.name.encode('unicode_escape'))

you need to encode with unicode_escape before saving as mention in above example (self.name = str(self.name.encode('unicode_escape'))), no need to do any other setting or db changes

user_47306640

Instead of handling on DB level i build my custom Field to handle encoding and decoding of data.

class BaseEmojiField:
    def to_python(self, value ):
        if isinstance(value, str) or value is None:
            if value is None:
                return value
            else:
                encoded_string = str(value).encode('unicode_escape')
                return encoded_string
        return str(value)
    
    def from_db_value(self, value, *args, **kwargs):
        if str(value).startswith("b'"):
            value = bytes(str(value).replace("b'","")[0:-1].replace('\\\\','\\'), 'utf-8').decode('unicode_escape')
        elif str(value).isascii():
            value = bytes(str(value), 'utf-8').decode('unicode_escape')
            return value
        return value

Then you can use it with both Charfield and TextField

class CustomEmojiCharField(BaseEmojiField , models.CharField):
    pass

class CustomEmojiTextField(BaseEmojiField ,models.TextField):
    pass

so where you need emoji use these fields instead of models.CharField and models.TextField Also there is one drawback that if you edit anything from django admin site it goes int django admin log. admin log uses str method to retreive information of your instance. so in str method dont use field with emoji/special character.

user_66876980

I think that the problem is with emojies inside the text you are posting.

I had RichTextUploadingField for a field and when I put emojies inside the field, it returns the error you have. I just removed the emojies and it works fine!

user_59475260
Related