Writing to SQL yields different results with shell / django

I have a simple model:

class myJSONField(models.JSONField):
    def get_prep_value(self, v):
        if v is None:
            return value
        return json.dumps(v, ensure_ascii = False)

class Product(models.Model):
    data                = myJSONField()
    number              = models.PositiveIntegerField()

Product.create(number = 1, data = {"direction": "süd"}) yields:

"{\"direction\": \"s\u00fcd\"}"

Testing with sqlite3:

conn = sqlite3.connect('db.sqlite3')
c = conn.cursor()
test = json.dumps({"direction": "Süd"}, ensure_ascii = False)
sql = ("""UPDATE test_product SET data = ? WHERE id=1""")
c.execute(sql, [test])
conn.commit()

yields:

{"direction": "süd"}

I am using the same db.sqlite3 file for both tests.

1. I do not understand why the ensure_ascii = False flag is ignored by the django logic.
2. How can I obtain a correct JSON in my JSONField (without the escaped ")?

Edit: I can see the result in my django admin and I also tested omitting the ensure_ascii = False setting in SQLite:

a = {"direction": "Süd", }
b = json.dumps(a)#, ensure_ascii = False)
sql = ("""UPDATE test_product SET data = ? WHERE id=1""")
c.execute(sql, [b])
conn.commit()

which leads to:

{"direction": "S\u00fcd"}

(remark: no " around that one!)

When writing to the database with Django and reading via python manage.py shell and sqlite3 directly:

import sqlite3
conn = sqlite3.connect('db.sqlite3')
c = conn.cursor()
c.execute('SELECT * FROM virtualShelf_product WHERE id=1')
res = c.fetchall()


[(1, 1, '"{\\"direction\\": \\"s\\u00fcd\\"}"')]

when reading the same product via Django ORM:

p = Product.objects.get(id=1)
p.data
'{"direction": "süd"}'

By overriding get_prep_value, you are serializing twice. Indeed, once in your myJSONField, and a second time by the get_db_prep_value function which uses get_prep_value as a helper function.

Indeed, if we inspect the source code ^[GitHub], we see:

> <pre><code>def get_db_prep_value(self, value, connection, prepared=False):
> if not prepared:
> value = self.get_prep_value(value)
> # &hellip;
> return <b>connection.ops.adapt_json_value(value, self.encoder)</b></code></pre>

depending on the connection itself, Django can then send for example a BSON blob, or a JSON. Except for PostgreSQL, it encodes the data with&nbsp;^[GitHub]:

<pre><code>def adapt_json_value(self, value, encoder):
return <b>json.dumps(</b>value, cls=encoder<b>)</b></code></pre>

if you override get_prep_value, it converts it to a string, and then you JSON serialize it a second time.

One can implement a custom backend that would pass ensure_ascii=False, a poor man's solution would be to just inject this through monkey patching. For example in the AppConfig:

<pre><code># <em>app_name</em>/apps.py

from django.apps import AppConfig

class AppConfig(AppConfig):
name = &quot;<em>app_name</em>&quot;

def ready(self):
    def adapt_json_value(self, value, encoder):
        return json.dumps(value, <b>ensure_ascii=False</b>, cls=encoder)

    from django.db.backends.base.operations import BaseDatabaseOperations

    BaseDatabaseOperations<b>.adapt_json_value = adapt_json_value</b></code></pre>

of course we then omit the get_prep_value in the myJSONField, which also basically nullifies myJSONField over JSONField.