PostgreSQL: The Versatile INSERT

Inserting a single row into a table is what comes to mind when you think of the INSERT statement in PostgreSQL. It does, however, have a few more tricks up it’s sleeve! Read on to discover some of the more interesting things you can do with INSERT.

Copying in Bulk

Let’s say you want to periodically capture snapshots of a table – all rows in the table should be copied to another table, with an additional timestamp column denoting when the snapshot was taken. Here’s how you can create and populate the table the first time around:

demo=# SELECT * FROM mytable;
 ticker | quote
--------+-------
 FOO    | $4.01
 BAR    | $1.42
(2 rows)

demo=# CREATE TABLE snaps_of_mytable AS
demo-#   SELECT current_timestamp AS snapped_at, *
demo-#     FROM mytable;
SELECT 2
demo=#
demo=# SELECT * FROM snaps_of_mytable ;
         snapped_at          | ticker | quote
-----------------------------+--------+-------
 2018-10-09 04:16:22.3613+00 | FOO    | $4.01
 2018-10-09 04:16:22.3613+00 | BAR    | $1.42
(2 rows)

And from then on, you can use the INSERT..SELECT form of INSERT statement to copy rows from one table and insert into another. You can fill in extra values into the destination table row too.

demo=# INSERT INTO snaps_of_mytable
demo-#   SELECT current_timestamp AS snapped_at, *
demo-#     FROM mytable;
INSERT 0 2
demo=#
demo=# SELECT * FROM snaps_of_mytable ;
          snapped_at           | ticker | quote
-------------------------------+--------+-------
 2018-10-09 04:16:22.3613+00   | FOO    | $4.01
 2018-10-09 04:16:22.3613+00   | BAR    | $1.42
 2018-10-09 04:18:53.432224+00 | BAR    | $1.42
 2018-10-09 04:18:53.432224+00 | FOO    | $4.10
(4 rows)

Upserts

In PostgreSQL 9.5, the ON CONFLICT clause was added to INSERT. This lets application developers write less code and do more work in SQL.

Here is a table of key, value pairs:

demo=# SELECT * FROM kv;
 key  |   value
------+-----------
 host | 127.0.0.1
 port | 5432
(2 rows)

A common use case is to insert a row only if it does not exist – and if it does, do not overwrite. This is done with the ON CONFLICT..DO NOTHING clause of the INSERT statement:

demo=# INSERT INTO kv (key, value) VALUES ('port', '3306')
demo-# ON CONFLICT (key) DO NOTHING;
INSERT 0 0
demo=# SELECT * FROM kv;
 key  |   value
------+-----------
 host | 127.0.0.1
 port | 5432
(2 rows)

Another common usage is to insert a row if it does not exist, and update the value, if it does. This can be done with the ON CONFLICT..DO UPDATE clause.

demo=# INSERT INTO kv (key, value) VALUES ('host', '10.0.10.1')
demo-# ON CONFLICT (key) DO UPDATE SET value=EXCLUDED.value;
INSERT 0 1
demo=# INSERT INTO kv (key, value) VALUES ('ssl', 'off')
demo-# ON CONFLICT (key) DO UPDATE SET value=EXCLUDED.value;
INSERT 0 1
demo=# SELECT * FROM kv;
 key  |   value
------+-----------
 host | 10.0.10.1
 port | 5432
 ssl  | off
(3 rows)

In the first case above the value of ‘host’ was overwritten with the new value, and in the second case the value of ‘ssl’ was inserted as the third row.

Even more sophisticated use cases can be realized with DO UPDATE. Consider the table below, where in addition to key and value, there is a column called “accumulate”. For rows where accumulate is true, the values are meant to be accumulated as a comma-separated string. For other rows, values are single-valued.

demo=# CREATE TABLE kv2 (
demo(#     key text PRIMARY KEY,
demo(#     accumulate boolean NOT NULL DEFAULT false,
demo(#     value text
demo(# );
CREATE TABLE
demo=# INSERT INTO kv2 VAlUES
demo-#     ('port', false, '5432'),
demo-#     ('listen', true, NULL);
INSERT 0 2
demo=# SELECT * FROM kv2;
  key   | accumulate | value
--------+------------+-------
 port   | f          | 5432
 listen | t          |
(2 rows)

The WHERE clause can be used to either overwrite the “value” column, or append into it, depending on the value of “accumulate”, like this:

demo=# INSERT INTO kv2 AS t (key, value) VALUES ('port', '3306')
demo-# ON CONFLICT (key) DO UPDATE SET value = concat_ws(',', t.value, EXCLUDED.value)
demo-# WHERE t.accumulate;
INSERT 0 0
demo=# INSERT INTO kv2 AS t (key, value) VALUES ('listen', '127.0.0.1')
demo-# ON CONFLICT (key) DO UPDATE SET value = concat_ws(',', t.value, EXCLUDED.value)
demo-# WHERE t.accumulate;
INSERT 0 1
demo=# INSERT INTO kv2 AS t (key, value) VALUES ('listen', '10.0.10.1')
demo-# ON CONFLICT (key) DO UPDATE SET value = concat_ws(',', t.value, EXCLUDED.value)
demo-# WHERE t.accumulate;
INSERT 0 1
demo=# SELECT * FROM kv2;
  key   | accumulate |        value
--------+------------+---------------------
 port   | f          | 5432
 listen | t          | 127.0.0.1,10.0.10.1
(2 rows)

The first statement did not accumulate the value of ‘3306’ into ‘port’ because ‘accumulate’ was off for that row. The next two statements added the values ‘127.0.0.1’ and ‘10.0.10.1’ into the value of ‘listen’, because ‘accumulate’ was true.

Returning Generated Values

Values generated by PostgreSQL during insert, like default values or autoincremented SERIAL values can be returned using the RETURNING clause of the INSERT statement.

Assume you need to generate random UUIDs as keys for rows in a table. You can let PostgreSQL do the work of generating the UUIDs and have it return the generated value to you like this:

demo=# INSERT INTO kv (key, value) VALUES (gen_random_uuid(), 'foo') RETURNING key;
                 key
--------------------------------------
 d93ceaa5-30a8-4285-83c5-7defa79e2f90
(1 row)

INSERT 0 1
demo=# INSERT INTO kv (key, value) VALUES (gen_random_uuid(), 'bar') RETURNING key;
                 key
--------------------------------------
 caf9c5d9-9a79-4b26-877f-a75a083b0c79
(1 row)

INSERT 0 1
demo=# SELECT * FROM kv;
                 key                  | value
--------------------------------------+-------
 d93ceaa5-30a8-4285-83c5-7defa79e2f90 | foo
 caf9c5d9-9a79-4b26-877f-a75a083b0c79 | bar
(2 rows)

Moving Rows with CTE Clauses

You can even move rows between tables with INSERT, using the WITH clause. Here are two tables with todo lists for different years.

demo=# SELECT * FROM todos_2018;
      what      | done
----------------+------
 thing to do #1 | t
 thing to do #2 | t
 thing to do #3 | f
(3 rows)

demo=# SELECT * FROM todos_2019;
 what | done
------+------
(0 rows)

To move the todo items that are not yet completed in 2018 to 2019, you can basically delete such rows from the 2018 table and insert them into the 2019 table in one shot:

demo=# WITH items AS (
demo(#     DELETE FROM todos_2018
demo(#     WHERE NOT done
demo(#     RETURNING *
demo(# )
demo-# INSERT INTO todos_2019 SELECT * FROM items;
INSERT 0 1
demo=# SELECT * FROM todos_2018;
      what      | done
----------------+------
 thing to do #1 | t
 thing to do #2 | t
(2 rows)

demo=# SELECT * FROM todos_2019;
      what      | done
----------------+------
 thing to do #3 | f
(1 row)

To learn more about the smart little INSERT statement, check out the documentation and experiment!

About pgDash

pgDash is a modern, in-depth monitoring solution designed specifically for PostgreSQL deployments. pgDash shows you information and metrics about every aspect of your PostgreSQL database server, collected using the open-source tool pgmetrics. pgDash provides core reporting and visualization functionality, including collecting and displaying PostgreSQL information and providing time-series graphs, detailed reports, alerting, teams and more.

The Diagnostics feature in pgDash examines your PostgreSQL server and databases scanning for potential issues that can impact the health and performance of the deployment. No additional setup is required for Diagnostics - you just need to be actively sending in data to pgDash. Learn more here or signup today for a free trial.