Android Room Persistence Library: Upsert

Viewed 70104

Android's Room persistence library graciously includes the @Insert and @Update annotations that work for objects or collections. I however have a use case (push notifications containing a model) that would require an UPSERT as the data may or may not exist in the database.

Sqlite doesn't have upsert natively, and workarounds are described in this SO question. Given the solutions there, how would one apply them to Room?

To be more specific, how can I implement an insert or update in Room that would not break any foreign key constraints? Using insert with onConflict=REPLACE will cause the onDelete for any foreign key to that row to be called. In my case onDelete causes a cascade, and reinserting a row will cause rows in other tables with the foreign key to be deleted. This is NOT the intended behavior.

12 Answers

Perhaps you can make your BaseDao like this.

secure the upsert operation with @Transaction, and try to update only if insertion is failed.

@Dao
public abstract class BaseDao<T> {
    /**
    * Insert an object in the database.
    *
     * @param obj the object to be inserted.
     * @return The SQLite row id
     */
    @Insert(onConflict = OnConflictStrategy.IGNORE)
    public abstract long insert(T obj);

    /**
     * Insert an array of objects in the database.
     *
     * @param obj the objects to be inserted.
     * @return The SQLite row ids   
     */
    @Insert(onConflict = OnConflictStrategy.IGNORE)
    public abstract List<Long> insert(List<T> obj);

    /**
     * Update an object from the database.
     *
     * @param obj the object to be updated
     */
    @Update
    public abstract void update(T obj);

    /**
     * Update an array of objects from the database.
     *
     * @param obj the object to be updated
     */
    @Update
    public abstract void update(List<T> obj);

    /**
     * Delete an object from the database
     *
     * @param obj the object to be deleted
     */
    @Delete
    public abstract void delete(T obj);

    @Transaction
    public void upsert(T obj) {
        long id = insert(obj);
        if (id == -1) {
            update(obj);
        }
    }

    @Transaction
    public void upsert(List<T> objList) {
        List<Long> insertResult = insert(objList);
        List<T> updateList = new ArrayList<>();

        for (int i = 0; i < insertResult.size(); i++) {
            if (insertResult.get(i) == -1) {
                updateList.add(objList.get(i));
            }
        }

        if (!updateList.isEmpty()) {
            update(updateList);
        }
    }
}

For more elegant way to do that I would suggest two options:

Checking for return value from insert operation with IGNORE as a OnConflictStrategy (if it equals to -1 then it means row wasn't inserted):

@Insert(onConflict = OnConflictStrategy.IGNORE)
long insert(Entity entity);

@Update(onConflict = OnConflictStrategy.IGNORE)
void update(Entity entity);

@Transaction
public void upsert(Entity entity) {
    long id = insert(entity);
    if (id == -1) {
        update(entity);   
    }
}

Handling exception from insert operation with FAIL as a OnConflictStrategy:

@Insert(onConflict = OnConflictStrategy.FAIL)
void insert(Entity entity);

@Update(onConflict = OnConflictStrategy.FAIL)
void update(Entity entity);

@Transaction
public void upsert(Entity entity) {
    try {
        insert(entity);
    } catch (SQLiteConstraintException exception) {
        update(entity);
    }
}

If the table has more than one column, you can use

@Insert(onConflict = OnConflictStrategy.REPLACE)

to replace a row.

Reference - Go to tips Android Room Codelab

This is the code in Kotlin:

@Insert(onConflict = OnConflictStrategy.IGNORE)
fun insert(entity: Entity): Long

@Update(onConflict = OnConflictStrategy.REPLACE)
fun update(entity: Entity)

@Transaction
fun upsert(entity: Entity) {
  val id = insert(entity)
   if (id == -1L) {
     update(entity)
  }
}

I found an interesting reading about it here.

It is the "same" as posted on https://stackoverflow.com/a/50736568/4744263. But, if you want an idiomatic and clean Kotlin version, here you go:

    @Transaction
    open fun insertOrUpdate(objList: List<T>) = insert(objList)
        .withIndex()
        .filter { it.value == -1L }
        .forEach { update(objList[it.index]) }

    @Insert(onConflict = OnConflictStrategy.IGNORE)
    abstract fun insert(obj: List<T>): List<Long>

    @Update
    abstract fun update(obj: T)

Alternatively to make UPSERT manually in loop like it's suggested in @yeonseok.seo post, we may use UPSERT feature provided by Sqlite v.3.24.0 in Android Room.

Nowadays, this feature is supported by Android 11 and 12 with default Sqlite version 3.28.0 and 3.32.2 respectively. If you need it in versions prior Android 11 you can replace default Sqlite with custom Sqlite project like this https://github.com/requery/sqlite-android (or built your own) to have this and other features that are available in latest Sqlite versions, but not available in Android Sqlite provided by default.

If you have Sqlite version starting from 3.24.0 on device, you can use UPSERT in Android Room like this:

@Query("INSERT INTO Person (name, phone) VALUES (:name, :phone) ON CONFLICT (name) DO UPDATE SET phone=excluded.phone")
fun upsert(name: String, phone: String)

Another approach I can think of is to get the entity via DAO by query, and then perform any desired updates. This may be less efficient compared to the other solutions in this thread in terms of runtime because of having to retrieve the full entity, but allows much more flexibility in terms of operations allowed such as on what fields/variable to update.

For example :

private void upsert(EntityA entityA) {
   EntityA existingEntityA = getEntityA("query1","query2");
   if (existingEntityA == null) {
      insert(entityA);
   } else {
      entityA.setParam(existingEntityA.getParam());
      update(entityA);
   }
}

Here is a way to use a real UPSERT clause in Room library.

The main advantage of this method is that you can update rows for which you don't know their ID.

  1. Setup Android SQLite support library in your project to use modern SQLite features on all devices:
  2. Inherit your daos from BasicDao.
  3. Probably, you want to add in your BasicEntity: abstract fun toMap(): Map<String, Any?>

Use UPSERT in your Dao:

@Transaction
private suspend fun upsert(entity: SomeEntity): Map<String, Any?> {
    return upsert(
        SomeEntity.TABLE_NAME,
        entity.toMap(),
        setOf(SomeEntity.SOME_UNIQUE_KEY),
        setOf(SomeEntity.ID),
    )
}
// An entity has been created. You will get ID.
val rawEntity = someDao.upsert(SomeEntity(0, "name", "key-1"))

// An entity has been updated. You will get ID too, despite you didn't know it before, just by unique constraint!
val rawEntity = someDao.upsert(SomeEntity(0, "new name", "key-1"))

BasicDao:

import android.database.Cursor
import androidx.room.*
import androidx.sqlite.db.SimpleSQLiteQuery
import androidx.sqlite.db.SupportSQLiteQuery

abstract class BasicDao(open val database: RoomDatabase) {
    /**
     * Upsert all fields of the entity except those specified in [onConflict] and [excludedColumns].
     *
     * Usually, you don't want to update PK, you can exclude it in [excludedColumns].
     *
     * [UPSERT](https://www.sqlite.org/lang_UPSERT.html) syntax supported since version 3.24.0 (2018-06-04).
     * [RETURNING](https://www.sqlite.org/lang_returning.html) syntax supported since version 3.35.0 (2021-03-12).
     */
    protected suspend fun upsert(
        table: String,
        entity: Map<String, Any?>,
        onConflict: Set<String>,
        excludedColumns: Set<String> = setOf(),
        returning: Set<String> = setOf("*")
    ): Map<String, Any?> {
        val updatableColumns = entity.keys
            .filter { it !in onConflict && it !in excludedColumns }
            .map { "`${it}`=excluded.`${it}`" }

        // build sql
        val comma = ", "
        val placeholders = entity.map { "?" }.joinToString(comma)
        val returnings = returning.joinToString(comma) { if (it == "*") it else "`${it}`" }
        val sql = "INSERT INTO `${table}` VALUES (${placeholders})" +
                " ON CONFLICT(${onConflict.joinToString(comma)}) DO UPDATE SET" +
                " ${updatableColumns.joinToString(comma)}" +
                " RETURNING $returnings"

        val query: SupportSQLiteQuery = SimpleSQLiteQuery(sql, entity.values.toTypedArray())
        val cursor: Cursor = database.openHelper.writableDatabase.query(query)

        return getCursorResult(cursor).first()
    }

    protected fun getCursorResult(cursor: Cursor, isClose: Boolean = true): List<Map<String, Any?>> {
        val result = mutableListOf<Map<String, Any?>>()
        while (cursor.moveToNext()) {
            result.add(cursor.columnNames.mapIndexed { index, columnName ->
                val columnValue = if (cursor.isNull(index)) null else cursor.getString(index)
                columnName to columnValue
            }.toMap())
        }

        if (isClose) {
            cursor.close()
        }
        return result
    }
}

Entity example:

import androidx.room.ColumnInfo
import androidx.room.Entity
import androidx.room.Index
import androidx.room.PrimaryKey

@Entity(
    tableName = SomeEntity.TABLE_NAME,
    indices = [Index(value = [SomeEntity.SOME_UNIQUE_KEY], unique = true)]
)
data class SomeEntity(
    @PrimaryKey(autoGenerate = true)
    @ColumnInfo(name = ID)
    val id: Long,

    @ColumnInfo(name = NAME)
    val name: String,

    @ColumnInfo(name = SOME_UNIQUE_KEY)
    val someUniqueKey: String,
) {
    companion object {
        const val TABLE_NAME = "some_table"
        const val ID = "id"
        const val NAME = "name"
        const val SOME_UNIQUE_KEY = "some_unique_key"
    }

    fun toMap(): Map<String, Any?> {
        return mapOf(
            ID to if (id == 0L) null else id,
            NAME to name,
            SOME_UNIQUE_KEY to someUniqueKey
        )
    }
}

Should be possible with this sort of statement:

INSERT INTO table_name (a, b) VALUES (1, 2) ON CONFLICT UPDATE SET a = 1, b = 2

If you have legacy code: some entities in Java and BaseDao as Interface (where you cannot add a function body) or you too lazy for replacing all implements with extends for Java-children.

Note: It works only in Kotlin code. I'm sure that you write new code in Kotlin, I'm right? :)

Finally a lazy solution is to add two Kotlin Extension functions:

fun <T> BaseDao<T>.upsert(entityItem: T) {
    if (insert(entityItem) == -1L) {
        update(entityItem)
    }
}

fun <T> BaseDao<T>.upsert(entityItems: List<T>) {
    val insertResults = insert(entityItems)
    val itemsToUpdate = arrayListOf<T>()
    insertResults.forEachIndexed { index, result ->
        if (result == -1L) {
            itemsToUpdate.add(entityItems[index])
        }
    }
    if (itemsToUpdate.isNotEmpty()) {
        update(itemsToUpdate)
    }
}
Related