Skip to content
Advertisement

Inserting data into psql database with high performance

Assume, I have a Python program, and I have an Offer object Offer(title='title1', category='cat1', regions=['reg1']). I want to add this Offer into psql db, with minimal number of queries (performance). Inserts of new regions and categories are rare (number of regions and categories is limited (and are unique), but number of offers is unlimited). Basically Regions and Categories can be inserted by query:

JavaScript

, but I need to execute another query to get id of region/category (when region/category already exists). I need this id to execute query inserting data to Offers table:

JavaScript

Currently my code looks like:

JavaScript

I don’t know how to do it robust and efficient (without unnecessary SELECTs). I work with Python/psycopg2.


Tables:

JavaScript

Advertisement

Answer

You are looking for INSERT ... ON CONFLICT DO NOTHING.

For that, you need a unique constraint on the column that identifies the object.

That allows you to retrieve the generated id, for example:

JavaScript

Dependent tables could be filled like this, using a variable cat_id that was set with the result from the above query:

JavaScript

There is of course a race condition: someone could delete the categories row before your second INSERT. But perhaps that is good enough.

User contributions licensed under: CC BY-SA
7 People found this is helpful
Advertisement