How can I do 'insert if not exists' in MySQL?

Viewed 997664

I started by googling and found the article How to write INSERT if NOT EXISTS queries in standard SQL which talks about mutex tables.

I have a table with ~14 million records. If I want to add more data in the same format, is there a way to ensure the record I want to insert does not already exist without using a pair of queries (i.e., one query to check and one to insert is the result set is empty)?

Does a unique constraint on a field guarantee the insert will fail if it's already there?

It seems that with merely a constraint, when I issue the insert via PHP, the script croaks.

11 Answers

Here is a PHP function that will insert a row only if all the specified columns values don't already exist in the table.

  • If one of the columns differ, the row will be added.

  • If the table is empty, the row will be added.

  • If a row exists where all the specified columns have the specified values, the row won't be added.

     function insert_unique($table, $vars)
     {
       if (count($vars)) {
         $table = mysql_real_escape_string($table);
         $vars = array_map('mysql_real_escape_string', $vars);
    
         $req = "INSERT INTO `$table` (`". join('`, `', array_keys($vars)) ."`) ";
         $req .= "SELECT '". join("', '", $vars) ."' FROM DUAL ";
         $req .= "WHERE NOT EXISTS (SELECT 1 FROM `$table` WHERE ";
    
         foreach ($vars AS $col => $val)
           $req .= "`$col`='$val' AND ";
    
         $req = substr($req, 0, -5) . ") LIMIT 1";
    
         $res = mysql_query($req) OR die();
         return mysql_insert_id();
       }
       return False;
     }
    

Example usage:

<?php
  insert_unique('mytable', array(
    'mycolumn1' => 'myvalue1',
    'mycolumn2' => 'myvalue2',
    'mycolumn3' => 'myvalue3'
    )
  );
?>

Something worth noting is that INSERT IGNORE will still increment the primary key whether the statement was a success or not just like a normal INSERT would.

This will cause gaps in your primary keys that might make a programmer mentally unstable. Or if your application is poorly designed and depends on perfect incremental primary keys, it might become a headache.

Look into innodb_autoinc_lock_mode = 0 (server setting, and comes with a slight performance hit), or use a SELECT first to make sure your query will not fail (which also comes with a performance hit and extra code).

Update or insert without known primary key

If you already have a unique or primary key, the other answers with either INSERT INTO ... ON DUPLICATE KEY UPDATE ... or REPLACE INTO ... should work fine (note that replace into deletes if exists and then inserts - thus does not partially update existing values).

But if you have the values for some_column_id and some_type, the combination of which are known to be unique. And you want to update some_value if exists, or insert if not exists. And you want to do it in just one query (to avoid using a transaction). This might be a solution:

INSERT INTO my_table (id, some_column_id, some_type, some_value)
SELECT t.id, t.some_column_id, t.some_type, t.some_value
FROM (
    SELECT id, some_column_id, some_type, some_value
    FROM my_table
    WHERE some_column_id = ? AND some_type = ?
    UNION ALL
    SELECT s.id, s.some_column_id, s.some_type, s.some_value
    FROM (SELECT NULL AS id, ? AS some_column_id, ? AS some_type, ? AS some_value) AS s
) AS t
LIMIT 1
ON DUPLICATE KEY UPDATE
some_value = ?

Basically, the query executes this way (less complicated than it may look):

  • Select an existing row via the WHERE clause match.
  • Union that result with a potential new row (table s), where the column values are explicitly given (s.id is NULL, so it will generate a new auto-increment identifier).
  • If an existing row is found, then the potential new row from table s is discarded (due to LIMIT 1 on table t), and it will always trigger an ON DUPLICATE KEY which will UPDATE the some_value column.
  • If an existing row is not found, then the potential new row is inserted (as given by table s).

Note: Every table in a relational database should have at least a primary auto-increment id column. If you don't have this, add it, even when you don't need it at first sight. It is definitely needed for this "trick".

INSERT INTO table_name (columns) VALUES (values) ON CONFLICT (id) DO NOTHING;
Related