Hacker News new | past | comments | ask | show | jobs | submit login

The .wal file comes and goes, so occasionally things like rsync will fail if the file no longer exists by the time it tries to transfer.



SQLITE_FCNTL_PERSIST_WAL can be used to avoid the .wal file being completely deleted. Also useful for when you need clients without write access to the database directory to always be able to open it read-only.


The .wal file shouldn't be backed up though.


If the .wal file is not backed up, then any information in it will not be backed up. Depending on the settings for the WAL, this can be quite a lot. You would need to do something like the VACUUM INTO trick described above if you want to safely backup the sqlite database from a live filesystem, because there is a risk of the WAL and database being out of sync. I’m not sure that SQLite, good though it is, can get around the WAL being taken at a different time to the rest of the database.

The problem here isn’t the .wal file, but doing backups from a moving filesystem. The solution is to use snapshots. ZFS supports snapshots natively, and Linux’s device mapper supports snapshots of any block device. [0]

[0] https://www.kernel.org/doc/html/latest/admin-guide/device-ma...


Why? An atomic filesystem snapshot is very similar to power loss. Since WAL is required to handle that scenario, it's just as important when backing up through snapshots.




Join us for AI Startup School this June 16-17 in San Francisco!

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: