Monday, November 24, 2014

speeding up tests with Postgres

Dear Lazyweb,

I'm developing a site using Django and Postgres. How can I get my tests to run a lot faster?

Answer: tell Postgres to use a tablespace in RAM. Selects, updates, and inserts will run at fast RAM speed vs slow disk speed.

Caveat: this doesn't work for writes. Postgres is designed to keep data reliable at all times. Any INSERT/UPDATE/DELETE gets written to a "write-ahead log" (WAL), so that if the database crashed it'll restore the data.  Since this is on a disk no matter what the tablespace, the above trick doesn't work without defeating the WAL.

Answer2: use UNLOGGED tables, TBD.

Reference:

Unlogged table performance in postgresql 9.1: (2011) "unlogged tables have shown an increase of output by 13~17%"; includes performance-oriented Postgres settings.

WAITING FOR 9.1 – UNLOGGED TABLES (2011): "that's really fast"

wal_buffers performance by Robert Haas (2012): pretty graphs

Monday, November 3, 2014

talk Thursday: Networking Tricks with the Linux Kernel

If you're in LA, come see me Thursday at the Unix Users meeting!

Everyone knows about networking. The Linux kernel provides a wide range of networking and other services, most of which are not generally known. In fact, a lot of what "everyone knows" is wrong or is lacking in detail. In this talk I'll give a technical survey of what's available and how to use it to accomplish magic tricks. With the right knowledge,our software can do great things without much code!