Hacker News new | past | comments | ask | show | jobs | submit login

Mr. Chu, I hope you never lose your tenacity with respect to writing blurbs on LMDB's performance and reliability. I have enjoyed the articles comparing LMDB with other databases' performance and hope you continue to point the spotlight on the superior design decisions of LMDB.



Thanks, glad you're getting something out of those writeups. Hopefully it helps other developers learn a better path.


What if another process calls fsync? It will get the error. Then when LMDB calls fsync no error will be reported. And thus the transaction will not be retried. Is this scenario dealt with?


Newer versions of linux (but not plenty of other OSs) guarantee that each write failure will be signalled once to each file descriptor open before the error occurred. So that ought to be handled, unless the FD is closed inbetween.


LMDB is single-writer. Only one process can acquire the writelock at a time, and only the writer can call fsync() so this scenario doesn't arise.

If you open the datafile with write access from another process without using the LMDB API, you're intentionally shooting yourself in the foot and all bets are off.


On some systems, fsync is system-wide. Another process fsyncing an unrelated file descriptor can still consume the error meant for the LMDB file descriptor. Same thing goes for a user running the sync command from the terminal. A write lock won't protect you from this, unless you can prevent all other processes from calling fsync. It's got nothing to do with opening the LMDB datafile concurrently. If you share a physical disk device with any other process, you're at risk.


Yes, acknowledged.

https://news.ycombinator.com/item?id=19128325

fsync() is not documented to be system-wide (while sync() is). That behavior would also be an OS bug. The question this person asked was specifically about fsync().


Sorry, that was confusing, I meant "system-wide" as in file system, not OS, i.e. "if you share a physical disk device with any other process".

When you flush a particular FD, some file systems just flush every dirty buffer for the entire disk. I wouldn't actually be surprised though if there are some kernels that flush all disks either, regardless of whether it's considered a bug or not.

"The question this person asked was specifically about fsync()."

Sure, but as you acknowledged elsewhere, if sync is called, it flushes everything, and this impacts upon the person who "asked specifically about fsync" since there's a chance on some kernels that sync will eat the error that fsync was expecting.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: