postgresql pgsql dynamic web link

Viewed 198

I am creating a recursive process to import csv data into a table. I can't seem to dynamically insert the variable I need to insert into my http request with my pgsql function.

I am trying to insert the variable 'insert_var' using %L with Execute Format and dynamic sql. When I see the link that the function passes it is says 'http://...variable=r1.insert_var....' instead of the actual variable stored in that column. What did I do wrong?

CREATE OR REPLACE FUNCTION recursive_data_import()
   RETURNS VOID AS $$

DECLARE 

insert_var text;

cur SCROLL CURSOR FOR
select * from symbols order by insert_var;
r1 RECORD;
BEGIN OPEN cur ;

LOOP

FETCH cur INTO r1;
EXIT WHEN NOT FOUND;

DROP TABLE IF EXISTS temp_table;

CREATE TABLE temp_table
(
    var1 numeric,
    var2 numeric,
);


EXECUTE format('COPY temp_table (var1, var2)
FROM PROGRAM 'wget -q -O - "http://website/Data?variable=%L&Type=0"' WITH (CSV), r1.insert_var);

ALTER TABLE temp_table
    ADD COLUMN var3 varchar(10);
UPDATE temp_table
       set var3 = r1.insert_var;

INSERT INTO final_table
(
SELECT 
    var1,
    var2,
    insert_var
    FROM temp_table
);

END LOOP;

CLOSE cur;

END; $$

LANGUAGE plpgsql;
2 Answers

looks like messy apostrophes in line

EXECUTE format('COPY temp_table (var1, var2)
FROM PROGRAM 'wget -q -O - "http://website/Data?variable=%L&Type=0"' WITH (CSV), r1.insert_var);

you can use a custom string separators:

EXECUTE format($x$COPY temp_table (var1, var2)
FROM PROGRAM 'wget -q -O - "http://website/Data?variable=%L&Type=0"' WITH (CSV)$x$, r1.insert_var);

I figured it out based on Pavel's help!

EXECUTE format($x$COPY temp_table (var1, var2)
FROM PROGRAM 'wget -q -O - "http://website/Data?variable=%I&Type=0"' WITH (FORMAT CSV)$x$, r1.insert_var);

Worked. Thank you!

Related