Understanding why queries slow down

In production environments with large databases and high concurrent access, it might happen that queries that used to run in tens of milliseconds suddenly take several seconds.

Likewise, a summary query for a report that used to run in a few seconds might take half an hour to complete.

Here are some ways to find out what is slowing them down.

Getting ready

Any questions of the type "why is this different today from what it was last week?" are much easier to answer if you have some kind of historical data collection setup.

Tools such as Cacti or Munin (for monitoring general server characteristics such as CPU and RAM usage, disk I/O, network traffic, and load average) are very useful to see what has changed recently, and to try to correlate these changes with the observed performance of some database operations.

Also, collecting historical statistics data from pg_stat_* tables—be that daily, hourly, or even every five minutes if you have enough disk space—is...