It's a really neat project. I did some simple benchmarks of it and it was between 10% slower to twice as slow. Which depending on your use case isn't that bad!
at segment we benchmarked https://github.com/segmentio/ctlstore against this driver. We saw about a 50% hit to read performance, so we didn't move forward with it, but the improvements in service build times were really appealing.
Honestly, on a non-toy project, build times with cgo are _brutal_. I agree with you usually, but when a build time on a beefy computer switches from under a second to >1min you notice it.
Linters and IDEs get slow when they check for errors, tests run slow, feedback drags, and all your workflows that took advantage of Go's fast compile times are now long enough that your flow and mental context disappear.
I'm way more lenient with other languages since the tooling and ecosystem are built around long build times. Your workflows compensate. But Go's tooling and ecosystem assume it compiles fast and treat things more like a scripting language. When that expectation is violated it hurts and everything feels like it's broken.
In my experience, encapsulating the access to sqlite in a go package helps a lot with avoiding recompilation of the c source, which indeed is brutally slow.
It acutally seems to be way slower than compiling with gcc from the command line. Anyone knows why this is the case?
If you have a lot of people and a lot of different builds it can become increasingly significant. It wasn't in our case, but it felt like it could have become so. And a lot of services wouldn't really notice too much if the occasional read from disk was slower, given that the nature of the data was control data.
I raised this question on twitter awhile back and the response that I got was that (for read-only tasks) the connection string might ameliorate this. I haven't been able to test it, though:
It’s hard for Go to beat C in a straight benchmark - Go is garbage collected and allocations often have extra overhead due to the memory layout of interfaces.
I was surprised with the announcement that Go's standard compiler had register allocation for function arguments[0] added to it. This seemed like table stakes for a very long time but it really illustrated just how Go's compiler is much, much simpler than most in many areas.
On one hand it makes you wonder how much could be squeezed out of Go and how many basic things they're still leaving out of the compiler. On the other it tells part of the story of why Go compiles fast with the backdrop that despite this lack of work by the compiler people use Go productively all day and deliver services they are happy with from a performance perspective.
It goes to show the impact of optimizing from top-down instead of bottom-up can have. Go was designed with performance in mind ("mechanical sympathy") as opposed to most other GC languages for which performance was an afterthought.
If you get high-level language constructs right, you'll get great performance by default, even with a simple compiler:
- structs instead of objects
- value types
- exposed pointers
- straightforward allocation semantics
Most compilers out there are busy fighting bad language designs. There's only so much a compiler can do when faced with fat objects hidden behind a spider web of pointers. So they have to do these microoptimizations to claw back at least some performance.
I wonder if they'll ever switch to a "normal = fast compile, low optimisation; release = slow compile, high optimisation" model like Rust. Although I suppose other languages already have the "high optimisation" parts covered (rust, julia, python+numpy, etc.) which might make it a fool's errand.
Gccgo at its core may be fast but when it comes to how the go frontend interpret it to gcc ir that pales in comparison than just straight up optimizating the Go compiler
There’ll more opportunities for more optimizations since they move to SSA ir since 1.5 and in turn compile time will definitely go down
Not only Go compiler is not as good at optimisation as C/C++/Rust/Zig/D compilers, but also Go FFI to C has significant overhead, which means there is often not much point in using low-level Linux APIs for performance advantage. Same problem like with Java. And in a database system you quite likely want a fine degree of control with non-standard stuff.
Other than what the other responders said, since it uses a C to Go compiler, I'd imagine the Go code that's generated isn't as fast as handwritten Go code could be.
https://datastation.multiprocess.io/blog/2022-05-12-sqlite-i...