How to convert Python decimal to SQLite numeric?

Question

I have a program that reads financial data in JSON and inserts it into an SQLite database. The problem is when I'm inserting it into SQLite numeric column and it doesn't seem to like the decimal object.

I've found this question answered before, but the answer is outdated and from what I understand SQLite now has a currency data type called numeric.

Right now as a workaround I'm storing decimal values as text, but is it possible to store it as numeric? Am I stuck with the overhead of converting decimals to strings and vice versa for database inserts and financial calculations?

user_2823719 · Answer

I am basically following the same approach as others but with one addition. The problem is that defining an adapter and converter takes care of the case of when you are accessing decimal columns row by row. But when you issue an aggregate function, such as summing a decimal column across all rows, the summation is done using real (i.e. floating point arithmetic) and the returned result will be a float with subsequent rounding errors.

The solution is to create a custom-tailored aggregate function, decimal_sum. Since sqlite3 aggregate functions are constrained in the types that they may return, decimal_sum will return a string representation of the decimal sum, which will be case to a Decimal type (this code is also Python 3-compatible):

import sqlite3
from decimal import Decimal


# DECTEXT columns will have TEXT affinity:
sqlite3.register_adapter(Decimal, lambda d: str(d))
sqlite3.register_converter("DECTEXT", lambda d: Decimal(d.decode('ascii')))


class DecimalSum:
    def __init__(self):
        self.sum = None

    def step(self, value):
        if value is None:
            return
        v = Decimal(value)
        if self.sum is None:
            self.sum = v
        else:
            self.sum += v

    def finalize(self):
        return None if self.sum is None else str(self.sum)


conn = sqlite3.connect(':memory:', detect_types=sqlite3.PARSE_DECLTYPES|sqlite3.PARSE_COLNAMES)
conn.create_aggregate("decimal_sum", 1, DecimalSum)

cursor = conn.cursor()
cursor.execute("""
    create table test (
        amount DECTEXT not null
    )
""")

for _ in range(1000):
    cursor.execute("insert into test(amount) values(?)", (Decimal("12.01"),))
conn.commit()

# Uses floating point math:
cursor.execute("select sum(amount) from test")
row = cursor.fetchone()
print('Floating point sum:', row[0], type(row[0]))

# Uses decimal math but returns result as a string
# and so we do a final conversion from string to Decimal:
cursor.execute("select decimal_sum(amount) as `amount [dectext]` from test")
row = cursor.fetchone()
print('Decimal sum:', row[0], type(row[0]))

cursor.close()
conn.close()

Prints:

Floating point sum: 12010.000000000178 
Decimal sum: 12010.00

How to convert Python decimal to SQLite numeric?

5 Answers