Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

My understanding is that that's not true, since if some other process (postgres or not) fsyncs the same file, then kernel will act as if it's been retried (because fsync called twice) so it will not report error correctly.


That's the bug in PostgreSQL. The fix is to not do the the fsync() in a different process that used an FD opened separately for the same file.


I think it's debatable whether that's a bug in PostgreSQL, an underspecified interface, or a bug somewhere else. A more interesting question is how we get it fixed.

The new Linux errseq_t design makes sure that every fd that was open before the error will see the error, and that at least one fd will see the error even if it happened when no one had it open (but only for as long as the inode doesn't fall out the cache). Before errseq_t came along, Linux was undeniably buggy here, since the AS_EIO flag could apparently be cleared in various ways and userspace could never be told about it.

The things achieved so far since the PostgreSQL community first crashed into this problem (thanks to the efforts of Craig Ringer):

* PostgreSQL now panics on failure, in cases where it previously retried (unless you set data_sync_retry = on, which should be safe on eg FreeBSD, though I don't think there is much point in it so the setting was included just as a matter of principle, when rolling out such a drastic change)

* Linux now reports errors to at least one fd in versions new enough to have errseq_t (in addition to reporting it in every fd that was open at the time); that came out of discussions between Linux and PostgreSQL people about all this

* There have also been changes to OpenBSD, though I'm not sure what exactly

* PostgreSQL hackers are working on a plan to make sure that file descriptors are held open until the data is synced, so that there is no reliance on the error state surviving in the inode cache during times when it's not open (this is complicated by the use of processes instead of threads)

* Longer term, this whole thing boosts interest in developing DIO support for PostgreSQL (previously thought to be a performance feature)


This is definitely debatable. However, because *BSD had the same semantics as Linux (IIUC), we can infer that the fsync()-syncs-only-writes-through-this-open-file semantics is actually what's reasonable to implement.


FreeBSD has had different semantics here for ~20 years: https://github.com/freebsd/freebsd/commit/e4e8fec98ae986357c...

I wish I could find the discussion around that commit.

It'll only start throwing away dirty buffers if the device actually goes away: https://github.com/freebsd/freebsd/commit/46b5afe7b1ae0ee655...




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: