Following a switchover from one node to another, it is common to think of performing a switchover back to the old master server, which is sometimes called failback or switchback.
Once a standby has become a master, it cannot go back to being a standby again. So, with log replication, there is no explicit switchback operation. This is a surprising situation for many people and is a repeated question, but it is quick to work around. Once you have performed a switchover, all you need to do is the following:
- Reconfigure the old master node again, repeating the same process as before to set up a standby node
- Switchover from the current to the old master node
The important part here is that if we perform the first step without deleting the files on the old master, it allows rsync to go much faster. When no files are present on the destination, rsync just performs a copy. When similarly named files are present on the destination, then rsync will compare the files and send only the changes. So, the rsync we perform on a switchback operation performs much less data transfer than in the original copy. It is likely that this will be enhanced in later releases of PostgreSQL. There are also ways to avoid this, as shown in the repmgr utility, which will be discussed later.
The pg_rewind utility has been developed as a way to perform an automated switchback operation. It performs a much faster switchback when there is a large database with few changes to apply. To allow correct operation, this program must run using the wal_log_hints = on parameter.
Using that parameter can cause more I/O on large databases, so while it improves performance for switchback it has a considerable overhead for normal running. If you think you would like to run pg_rewind, then make sure you work out how it behaves ahead of time. Trying to run it for the first time in a stress situation with a down server is a bad idea.