Unable to use utf8mb4 character set with CloudSQL on AppEngine Python

I have set up a CloudSQL instance that I am attempting to use with my Django app on AppEngine. I've confirmed that the server is set to use utf8mb4 character set via the CloudSQL console for my database:

utf8mb4 utf8mb4_unicode_ci

If I connect directly with the mysql cli, I can successfully insert and read emojis. However, if I insert the same emoji characters through the Django admin it's just inserted as "? ? ? ?".

I attempted to ensure the MySQLdb-python client is using utf8mb4 with:

'ENGINE': 'django.db.backends.mysql',
...
'OPTIONS': {
    'charset': "utf8mb4",
}

But this causes me to receive the following error on AppEngine:

(2019, "Can't initialize character set utf8mb4 (path: /usr/local/mysql/share/charsets/)")

My app.yaml is using the "latest" MySQLdb library:

libraries:
- name: MySQLdb
  version: "latest"