My personal DB solution actually averages 45 GiB / second on my laptop on a single thread and scales 1-1 with threads. It's way faster than SQLite. This is painfully mediocre performance.
You have to make engineering trade-offs for this, though. In my case, you can only recover 1 KiB records. It doesn't support compression. And you can only write to /dev/null.
To read data from this DB (which I built with command-line tools any Unix has), run:
dd if=/dev/zero of=/dev/null bs=1K count=${count}
You can vary the $count parameter to get more data.
You have to make engineering trade-offs for this, though. In my case, you can only recover 1 KiB records. It doesn't support compression. And you can only write to /dev/null.
To read data from this DB (which I built with command-line tools any Unix has), run:
You can vary the $count parameter to get more data.