I'm trying to do something in Postgres 12.5 and am either missing a detail, or pursuing a flawed strategy entirely. I have a simple lookup table with some longer strings and extended detail. As data flows into the main table, I'd like to convert those strings to a simple int for the key. I figured that I could keep the lookups in a table, and then use INSERT....ON CONFLICT DO NOTHING...RETURNING to get these outcomes:
- If the row does not exist in the lookup table, create it and return the new
id. <-- This works. - If the row does exist, don't overwrite it and return the new
id. <-- Theidis not returned.
The second case is by far the more common. Chances are, the system will run for days or week between new values hitting the lookup table. I prefer to do the INSERT assertively to avoid a concurrency issue or unique conflict. However, RETURNING doesn't work the way I'd hoped. I checked the docs, and the behavior I'm seeing is clearly documented. And, thinking about it, makes a lot of sense. So, moving forward, does it make more sense to write a function where I can test the outcome and issue a SELECT id FROM pgconfig_target WHERE... to recover the existing id, or is some other strategy more straightforward?
For background, the main table is a push_audit_log summarizing insert calls from our clients with timing, record counts, and other details. There are going to be lots of these records. That's why I'd rather shift the strings over to another table.
Here's a short version of the lookup table's setup:
--------------------------------------
-- Define table
------------------------------------
BEGIN;
DROP TABLE IF EXISTS dba.pgconfig_target CASCADE;
CREATE TABLE IF NOT EXISTS dba.pgconfig_target (
id int2 GENERATED ALWAYS AS IDENTITY,
schema_name citext NOT NULL DEFAULT NULL,
target_name citext NOT NULL DEFAULT NULL,
target_path citext NOT NULL DEFAULT NULL
);
ALTER TABLE dba.pgconfig_target
OWNER TO user_change_structure;
COMMIT;
------------------------------------
-- Build indexes
------------------------------------
CREATE UNIQUE INDEX pgconfig_target_unique_ix_btree
ON dba.pgconfig_target (schema_name, target_name, target_path);
And here's the INSERT I'm experimenting with:
INSERT INTO pgconfig_target (schema_name, target_name, target_path)
VALUES ('hello','world','passthrough.hello.world')
ON CONFLICT (schema_name,target_name,target_path) do nothing
RETURNING id;
Thanks for suggestions.
#Later# I'd like to use a suggestion like the (ever helpful) GMB listed, but can't seem to get it to work on the first run. I'm curious why, as there's something about order or execution that I'm not understanding.
I tried out a little scratch function, and it works, but a pure-SQL statement solution seems a bit nicer, if possible. Here's the function:
CREATE OR REPLACE FUNCTION ascendco.pgconfig_target_add_if_missing (
s_name citext,
t_name citext,
t_path citext
)
RETURNS integer AS
$BODY$
DECLARE
id_out integer = 0;
BEGIN
-- Insert the value, if it's missing.
INSERT INTO pgconfig_target (schema_name, target_name, target_path)
VALUES (s_name, t_name, t_path)
ON CONFLICT DO NOTHING;
-- The ID should be there, either historically or becase it was just added.
SELECT id
FROM pgconfig_target
WHERE schema_name = s_name
AND target_name = t_name
AND target_path = t_path
INTO id_out;
return id_out;
END
$BODY$
LANGUAGE plpgsql;
A sample call such as this works:
select * from pgconfig_target_add_if_missing ('hello', 'world', 'checking.it.out')
I'm using select * as I didn't sort out the syntax to make id an unambiguous result.