Long term, SQL will be phased out for new LLM-friendly query languages. The cost of AI generating tens of billions of queries per day creates a new incentive for rationality and efficiency that overcomes the human institutional weight of legacy languages. I'm working on such a language now: https://memelang.net/
I've only been through a few cycles, but there's three flavors I've seen:
1. Better query language: QUEL, LINQ, etc.
2. Better data model or performance: CouchDB, Mongo, Redis, etc.
3. Better abstraction: Zope, Active Record, et. al.
SQL vendors keep absorbing the differentiating features from other approaches with enough success that they capture most business use cases. That's not to say there aren't successes outside of SQL! But I've seen it claimed SQL will be dead several times over thanks to some new tech and most of the time SQL instead kills that tech or makes it niche.
I've always found it odd that most databases don't expose a kind of query bytecode API that SQL can be converted to. It would allow more kinds of languages, different SQL dialects, compile-time optimizations, and perhaps ORM could be compiled directly into your target database spec. Would reduce the amount of string operations by client code, at least
For translation between dialects, you could use projects such as SQLGlot. The advantage of SQL is the standardization over many decades (yes, I know that it's still a mess with the different dialects).
SQLGlot is amazing. In many ways it helps erase the differences and bridge between dialects. It is so useful for moving complex queries between platforms.
I do agree. SQL is simply an access API for so many systems, and nice as it's a declarative language rather than a normal programming language. LLMs are super powerful to express questions to data that can then be translated into SQL.
Cool idea! Although without looking closer I can't tell if "meme" is in reference to the technical or the colloquial meaning of meme.
Admittedly I don't know that much about LLM optimization/configuration, so apologies if I'm asking dumb questions. Isn't the value of needing to copy/paste that prompt in front of your queries a huge bog on net token efficiency? Like wouldn't you need to do some hundred/thousand query translations just to break even? Maybe I don't understand what you've built.
Thank you. That script prompt is just for development and exploration. A production model needs to be trained/fine-tuned on Memelang first. We're working on this now. The math says we can deliver a model 1/2 the size of an equivalent model for SQL.
Working on an AI-optimized query language. Like a terse, logical SQL. So smaller models can translate natural languages to DB queries more accurately. Saves lots of compute in RAG.
