Note that the variables are safely and correctly interpolated at compile time. And it's type checked across the boundary too, by checking (at compile time) the column types with the live database.
Yes, what you did is strictly more powerful than what the Python people did. And you did it 20 years ago. Well done, have an upvote. And yet, here we are in 2025 with Python popularity growing unstoppably and (approximately) no one caring about OCaml (and all the other languages better than Python). It makes me sad.
But my two cents is that we're pretty lucky it's python that has taken off like a rocket. It's not my favorite language, but there are far worse that it could have been.
I'm switching between C, OCaml, Python, bash & Rust roughly equally every day (to a lesser extent, Perl as well). Not everything is what gets on the front page of HN.
It's interesting how the majority has explicitly chosen NOT to use the "better" languages. Is the majority really that bad in their judgment? Or is it that "better" is actually defined by adoption over time?
It's clearly better in their opinion, they just aren't optimizing for the same metrics that you are. Python is better because it's easy for people to learn, imo.
If someone is challenged figuring out a venv and they're not an absolute beginner, perhaps they aren't cut out to work in technology. There are countless subjects in the field more challenging and complicated to wrap one's brain around.
gp's claim is not "its easy to learn". It's not just the concept -- it's the ergonomics, absolutely terrible footguns (especially when dealing with global wheels that can screw up your running system), and the hidden state.
When would you want to interface between a specific project and the global Python environment running your system? If there's ever a time when "lock the project into a venv and don't cross-contaminate its dependencies with the global Python environment" isn't the answer, that sounds like a corner case. Let each project be its thing by itself.
On modern Linux you can type `python` at the command prompt and get a REPL. On Windows you download an installer from the official website (just like one usually does to install anything on Windows), then use `py` at the command prompt.
You don't need to `import` anything to start teaching Python. Even then you can do quite a lot with the standard library. Even then, unless you're using 3.11 or later on Linux you can let Pip install with `--user` until you actually need to isolate things between projects. (And even with new Python on Linux, the instructor can typically avert this by just installing a separate Python in `/usr/local/bin` for example. Yes, that's "cheating", depending on the classroom environment. But that's part of the point: installation hurdles are hurdles for self-learners, not for students.)
You only need to learn about virtual environments once you have projects with mutually conflicting dependencies, and/or once you're at a point where you're ready to publish your own software and should be learning proper testing and development practices. (Which will be largely orthogonal to programming, and not trivial, in any language.)
Teachers teach Python because it's easy to teach while still being relevant to the real world, in particular because boilerplate is minimized. You don't have to explain jargon-y keywords like "public" or "static" up front. You don't have to use classes for quite some time (if ever, really). You can express iteration naturally. Types are naturally thought of in terms of capabilities.
In my mind, Python has all the pedagogical advantages of Lisp, plus enough syntactic cues to prevent getting "lost in a sea of parentheses". (Of course, it lacks plenty of other nice Lisp-family features.)
> In my mind, Python has all the pedagogical advantages of Lisp, plus enough syntactic cues to prevent getting "lost in a sea of parentheses". (Of course, it lacks plenty of other nice Lisp-family features.)
> Peter Norvig here. I came to Python not because I thought it was a better/acceptable/pragmatic Lisp, but because it was better pseudocode. Several students claimed that they had a hard time mapping from the pseudocode in my AI textbook to the Lisp code that Russell and I had online. So I looked for the language that was most like our pseudocode, and found that Python was the best match. Then I had to teach myself enough Python to implement the examples from the textbook. I found that Python was very nice for certain types of small problems, and had the libraries I needed to integrate with lots of other stuff, at Google and elsewhere on the net.
Basically, that it's better pedagogically because it looks like pseudo-code and it's easy to get up and running quickly.
Which is valid, but frustrating to see it lead to actual adoption outside of pedagogy. That property is entirely orthogonal to, almost at odds with, what makes a good programming language for medium to large production quality applications.
If we used that logic elsewhere in life we’d all be playing the flute and cycling around on tricycles and balance bikes. But for some reason in tech it’s all about Hello World.
The story of the winner being scrapy market entrants that are lower-cost (...of learning, in the case of python) and good-enough-quality (...than OCaml, Lisps, Haskel, definitely not JS or Java) is not a new one. I don't subscribe to your analogies.
> You don't have to explain jargon-y keywords like "public" or "static" up front.
patently not true. you dont get too far into python -- especially if you are reading (or copypastaing) other People's code -- before you see if __name__ == "__main__" and any potential future programmer will rightfully ask "what the absolute fuck is this"
even "def" is kind of a weird fucking keyword.
Don't get me started about teaching beginners which datatypes are pass by reference and which are pass by value.
try explaining to an elementary school student why
> Don't get me started about teaching beginners which datatypes are pass by reference and which are pass by value.
If they are beginners to programming, you wouldn't teach them those terms in the context of Python, because neither of those terms map to Python argument passing; Python has one form of argument passing, and it doesn't map closely to the intuition that experienced programmers in languages that have pass by reference and pass by value have about those things. Likewise, the only thing you'd teach someone new to Python that is experienced in languages where those terms are useful is that that distinction is irrelevant in Python, which is pass by assignment (sometimes also called pass by object reference, but pass by assignment is a much more useful description IMO, because argument passing works exactly like assignment to a new variable name.)
> try explaining to an elementary school student why
def foo(a):
a = a + 1
> doesn't change the caller's variable but
def bar(a):
a.append(1)
> does.
But, that's easy, if you've first taught them how variables, assignment, and mutation work in Python without getting function calls in the way, because it is exactly the same as this
a = 1
b = a
b = a + 1
print(f"{a=}, {b=}")
vs.
a = [1]
b = a
b.append[1]
print(f"{a=}, {b=}")
Argument passing is just assignment to a new variable that exists in the scope of the function. Methods which mutate an object affect the object no matter what variable you access it from, assignment operations affect only the variable they assign to. That's exactly the same behavior in one scope as it is between the function and caller scopes.
And this distinction has nothing to do with data types, but only with the operations performed (the only connection to data types is that immutable types have no mutation operations in the first place.) You can tell its not about data types because you can use the same types as the second excerpt, and the operations of the first, and get the same results as the first (which shares operations) and not the second (which shares datatypes):
a = [1]
b = a
b = b + [1]
print(f"{a=}, {b=}")
If you understand how assignment and mutation works in one scope, you understand how argument passing works. Trying to teach a distinction that exists between how different operations affect variables that initially reference the same object as a distinction about how datatypes are passed as arguments is confusing, because you as a teacher are presenting the source of the difference in behavior as originating in a completely different place than where it actually comes from. That's not a problem with Python being complex, it is a problem with you taking a very simple thing and making it complex by ascribing it to a source that is completely irrelevant to what is actually going on.
> you dont get too far into python -- especially if you are reading (or copypastaing) other People's code -- before you see if __name__ == "__main__"
First off, if you are teaching someone, you are showing that person the code, and not allowing copy-and-paste.
Second, no, that comes up much less often than you'd expect.
Third, it's the same as `if value == 'example':`. Underscores are not jargon.
Fourth, it's trivially searchable. That's the part where you can copy and paste - into a search engine, which will immediately find you several competent explanations such as https://stackoverflow.com/questions/419163 .
> even "def" is kind of a weird fucking keyword.
Admittedly a poor choice, but not a deal breaker. You need the concept of functions to do programming. But you don't need the concept of data hiding, nor do you need any of the multiple, barely-related concepts described by the term "static".
> Don't get me started about teaching beginners which datatypes are pass by reference and which are pass by value.
There's nothing to explain. They are all pass by value, per the strict meaning of those terms.
Those terms have been widely seen as less than ideal for decades, however, because they fail to account for variables with reference semantics (i.e., what Python uses - which are sometimes called "names"). A more modern term is "pass by assignment", which correctly describes all variable passing in Python: passing an argument to a parameter is a form of assignment, and works the same way as assigning a value to a name.
This is far less complex than C#, in which user-defined types may have either value semantics or reference semantics, and which supports both pass by assignment and two separate, true forms of pass by reference (for initialization and for modifying an existing object). And certainly it's less complex than whatever C++ is doing (see e.g. https://langdev.stackexchange.com/questions/3798 ).
> try explaining to an elementary school student why
First: if someone gives you a bag with three apples in it, you can put another apple in the bag and give it back, and the other person will have a bag with four apples in it. But if you add 3 + 1, that doesn't change the meaning of 3. These are simple ideas that an elementary school student already understands.
Second: from extensive experience teaching beginners (never mind that you are moving the goalposts now), it makes no sense to get into the theory. It's not helpful. A student who can ask about this has already lost the plot, because the two examples use completely different syntax (a method call versus an assignment) so they shouldn't be expected to work similarly. You avoid this problem by using more precise language early on. "Change" is not an appropriate word here.
Third: you give this example because you think that `bar` (and you imply by your naming that a list is being passed) demonstrates pass by reference. This is simply incorrect. Please read https://nedbatchelder.com/text/names1.html.
Fourth: your use of profanity and the overall tone of your writing suggests that you simply don't like the fact that Python works the way that it does. This is not a good look IMO.
Just for the record, I've been in variations of this discussion countless times. I know what I'm talking about. All links above are in my bookmarks.
In my experience people have to first figure out what the hell numpy is and how to get it (venv, conda, pip, uv, uvx, …) because python arrays are shit, and so people fix that wart with an external C library. Then they notice that some other dependency requires a previous python version, but their python is installed globally and other dependencies were installed for that. These are uniquely python-specific problems. Lisp doesn’t have those problems
Did they try using a search engine? But more to the point, if they don't understand what it is, how did they find out it exists?
> how to get it (venv, conda, pip, uv, uvx, …)
uvx is a command from the same program as uv; venv is not a way to obtain packages; and the choice here isn't a real stumbling block.
> because python arrays are shit
I can't say I've seen many people complain about the standard library `array` module; indeed it doesn't seem like many people are aware it exists in the first place.
If you're talking about lists then they serve a completely different purpose. But your use of profanity suggests to me that you don't have any actual concrete criticism here.
> Then they notice that some other dependency requires a previous python version
Where did this other dependency come from in the first place? How did they get here from a starting point of dissatisfaction with the Python standard library?
> but their python is installed globally and other dependencies were installed for that.
... and that's where actual environment management comes in, yes. Sometimes you have to do that. But this has nothing to do with teaching Python. You have necessarily learned quite a bit by the time this is a real concern, and if you were taught properly then you can self-study everything else you need.
> These are uniquely python-specific problems.
No other languages ever require environment management?
You don’t need to teach it to a beginner. The first of learning doesn’t need more than the standard library and when you need more than that you’re either giving them the single command necessary to run or, more likely, having them use a template project where a tool like Poetry is doing that automatically.
What this usually hits isn’t that managing Python packages is hard in 2025 but that many people do not learn how their operating system works conceptually until the first time they learn to program and it’s easy to conflate that with the first language you learn.
Let me introduce you to uv[0]. And yes it does say something that this tool isn't written in Python, but I'd say there's even more to be said that so many are trying to support Python.
yeah the reason why that's not an answer is because another half of users will say use poetry. if you want to do bioinformatics, people will insist on conda. then your team will say "use rye" and these strategies are somewhat compatible but ultimately mutually incompatible in ways that will drive you mad every time you hit a tiny snag that nonetheless grinds your attempt to just fucking run code to a halt.
It has become successful largely because it has always had really good foreign function interface. If you have a scientific or mathematical library laying around in C, then you could wire it up to Python, and then suddenly you have all the flexibility of a (fairly clean) scripting language to orchestrate your high speed C.
someone learning python as their first language knows so little its perfectly fine to let them pollute their global environment. Someone who knows other languages can understand what venv is for.
Instead they can type python to open a shell and use python to immediately run their file.
It's not, because you can fuck up system/unrelated app python dependencies and in extreme cases have to reinstall OS. Thankfully as developers migrate away from python/adopt strategies like flatpak this is less of a problem.
And yet that momentum and ecosystem wouldn't have been achieved in the first place if there weren't enough merits in the language to trigger and maintain that interest.
Some languages definitely had people gravitate towards them due to being innovative in a given space, but in many of those cases, the comparative advantage was lost to other languages/techs/frameworks that simply failed to gain a market share "equal to their innovative contribution" due to the first comer's advantage.
I think you're being too unfair. People aren't dumb.
It's also about how much better.
Beyond a decent enough type system, the advantages start to flatten and other factors start to matter more.
Can't speak too much for python, but as someone who's written large amounts of code in OCaml and Typescript, the strictest compiler options for Typescript are good enough.
No, people aren't dumb. They're practical. And so they choose to do what is practical, which in this case is to choose Python. And, to me, that makes it the better language.
Aren't there other benefits to server-side parameter binding besides just SQL-injection safety? For instance, using PG's extended protocol (binary) instead of just raw SQL strings. Caching parameterized prepared statements, etc.
Also:
db.execute(t"QUERY WHERE name = {name}")
Is dangerously close to:
db.execute(f"QUERY WHERE name = {name}")
A single character difference and now you've just made yourself trivially injectible.
I don't think this new format specifier is in any way applicable to SQL queries.
Templates are a very different duck type from strings and intentionally don't support __str__(). The SQL tool can provide a `safe_execute(Template)` that throws if passed a string and not a Template. You can imagine future libraries that only support Template and drop all functions that accept strings as truly safe query libraries.
> Caching parameterized prepared statements, etc.
Templates give you all the data you need to also build things like cacheable parameterized prepared statements. For DB engines that support named parameters you can even get the interpolation expression to auto-name parameters (get the string "name" from your example as the name of the variable filling the slot) for additional debugging/sometimes caching benefits.
You solve that with an execute(stmt) function that requires you to pass in a template.
In Javascript, sql`where id = ${id}` is dangerously close to normal string interpolation `where id = ${id}`, and db libs that offer a sql tag have query(stmt) fns that reject strings.
> A single character difference and now you've just made yourself trivially injectible.
No; a single character difference and now you get a `TypeError`, which hopefully the library has made more informative by predicting this common misuse pattern.
> Aren't there other benefits to server-side parameter binding besides just SQL-injection safety? For instance, using PG's extended protocol (binary) instead of just raw SQL strings. Caching parameterized prepared statements, etc.
All of which can be implemented on top of template strings.
> A single character difference and now you've just made yourself trivially injectible.
It's not just a one character difference, it's a different type. So `db.execute` can reject strings both statically and dynamically.
> I don't think
Definitely true.
> this new format specifier is in any way applicable to SQL queries.
The first mistake we're going to see a library developer make is:
def execute(query: Union[str, Template]):
Maybe because they want their execute function to be backwards compatible, or just because they really do want to allow either raw strings are a template string.
> they really do want to allow either raw strings are a template string.
I’d consider that an invalid use case:
1. You can create a template string without placeholders.
2. Even if the caller does need to pass in a string (because they’re executing from a file, or t-strings don’t support e.g. facetting) then they can just… wrap the string in a template explicitly.
I didn’t explicitly mention this in my post but, yes, the Template type is designed with caching in mind. In particular, the .strings tuple is likely to be useful as a cache key in many cases.
This is true of many other things, which is why we have type checkers and linters to be perfectly rigorous rather than expecting humans to never make mistakes.
Click "parent" a few times and look at the code example that started this thread. It's using the same function in a way that can't distinguish whether the user intentionally used a string (including an f-string) and a t-string.
Yes, and the parent is misguided. As was pointed out in multiple replies, the library can distinguish whether an ordinary string or a t-string is passed because the t-string is not a string instance, but instead creates a separate library type. A user who mistakenly uses an f prefix instead of a t prefix will, with a properly designed library, encounter a `TypeError` at runtime (or a warning earlier, given type annotations and a checker), not SQL injection.
In this particular instance it can't, because there are 3 ways in question here, and it can't distinguish between correct intentional usage and accidental usage of an f-string instead of a t-string:
db.execute("SELECT foo FROM bar;")
db.execute(f"SELECT foo FROM bar WHERE id = {foo_id};")
db.execute(t"SELECT foo FROM bar WHERE id = {foo_id};")
The first and second look identical to execute() because all it sees is a string. But the second one is wrong, a hard-to-see typo of the third.
If f-strings didn't exist there'd be no issue because it could distinguish by type as you say. But we have an incorrect SQL-injection-prone usage here that can't be distinguished by type from the correct plain string usage.
My (and their) point is that's the already existing API. You're proposing a big breaking change, with how many frameworks and tutorials are built on top of that.
It's not like this is the first time APIs have been improved. There are many tools (e.g. deprecation warnings & hints in editors, linter rules) that can help bridge the gap - even if t-strings are only used for new or refactored code, it's still a big improvement!
There's also simply no hard requirement to overload an `execute` function. We have options beyond "no templates at all" and "execute takes templates and strings", for example by introducing a separate function. Why does perfect have to be the enemy of good here?
Because they're both passed to "execute", which can't tell between the f-string and a non-interpolated query, so it just has to trust you did the right thing. Typoing the "t" as an "f" introduces SQL injection that's hard to spot.
Assuming `execute` takes both. You could have `execute(template)` and `execute_interpolated(str, ...args)` but yeah if it takes both you'll have challenges discouraging plain-text interpolation.
It would have to be the other way around or be a (possibly major) breaking change. Just execute() with strings is already standard python that all the frameworks build on top of, not to mention tutorials:
> It would have to be the other way around or be a (possibly major) breaking change.
If it is going to reject the currently-accepted unsafe usage, its going to be a major breaking change in any case, so I don't see the problem. I mean, if you are lamenting it can't reject the currently-accepted SQL-interpolated-via-f-string because it can't distinguish it by type from plain strings with no interpolation, you are already saying that you want a major breaking change but are upset because the particular implementation you want is not possible. So you can't turn around and dismiss an alternative solution because it would be a major breaking change, that's what was asked for!
With t-strings you could run proper escaping over the contents of `some_file` before passing it to a shell.
I'd have to take a look at the order things happen in shell, but you might even be able to increase security/foot-gun-potential a little bit here by turning this into something like `stat "$( base64 -d [base64 encoded content of some_file] )"`.
Oh! I missed this one because I've been looking specifically at the Packaging forum rather than the PEPs forum. This looks like a brilliant use case. (I'm aiming for wide compatibility - back to 3.6 - with my current projects, but I look forward to trying this out if and when it's accepted and implemented.)
Now if only the overall `subprocess` interface weren't so complex....
The key point is that t-strings are not strings. Db.execute(t”…”) would throw an exception, because t”…” is not a string and cannot be interpreted as one.
In order for a library to accept t-strings, they need to make a new function. Or else change the behavior and method signature of an old function, which I guess they could do but any sanely designed library doesn’t do.
Handling t-strings will require new functions to be added to libraries.
> The problem is writing f instead of t => and this will silently work I assume (not a Python dev just trying to understand the language design)
In the fullness of time it has no reason to. Even in the worst case scenario where you have to compose the query dynamically in a way t-strings can’t support, you can just instantiate a Template object explicitely.
>yes but the bug is writing f instead of t and I assume f will just work
but it will not. f-strings and t-strings are not compatible types, they will not "just work". not unless somebody changes a library to make it just work. as long as nobody does that, it's not an issue.
You add a new API that takes templates only leaving the existing API in place. You (some releases later) deprecate the string API. You (some releases later, with clear advance warning of when it is coming) actually remove the deprecated API. "It's a big breaking change around a brand new feature", yeah, so you don't break anything around a brand new feature, it's not like this kind of transition is a new concept.
The execute function can recognize it as a t-string and prevent SQL injection if the name is coming from user input. f-strings immediately evaluate to a string, whereas t-strings evaluate to a template object which requires further processing to turn it into a string.
Then the useful part is the extra execute function you have to write (it's not just a substitute like in the comment) and an extra function can confirm the safety of a value going into a f-string just as well.
I get the general case, but even then it seems like an implicit anti-pattern over doing db.execute(f"QUERY WHERE name = {safe(name)}")
Problem with that example is where do you get `safe`? Passing a template into `db.execute` lets the `db` instance handle safety specifically for the backend it's connected to. Otherwise, you'd need to create a `safe` function with a db connection to properly sanitize a string.
And further, if `safe` just returns a string, you still lose out on the ability for `db.execute` to pass the parameter a different way -- you've lost the information that a variable is being interpolated into the string.
db.safe same as the new db.execute with safety checks in it you create for the t-string but yes I can see some benefits (though I'm still not a fan for my own codebases so far) with using the values further or more complex cases than this.
Yeah but it would have to be something like `db.safe("SELECT * FROM table WHERE id = {}", row_id)` instead of `db.execute(t"SELECT * FROM table WHERE id = {row_id}")`.
This is just extra boilerplate though, for what purpose?.
I think one thing you might be missing is that in the t-string version, `db.execute` is not taking a string; a t-string resolves to an object of a particular type. So it is doing your `db.safe` operation, but automatically.
Of course you can write code like that. This is about making it easier not to accidentally cause code injection by forgetting the call of safe(). JavaScript had the same feature and some SQL libraries allow only the passing of template strings, not normal strings, so you can't generate a string with code injection. If you have to dynamically generate queries they allow that a parameter is another template string and then those are merged correctly. It's about reducing the likelihood of making mistakes with fewer key strokes. We could all just write untyped assembly instead and could do it safely by paying really good attention.
agreed. but then you're breaking the existing `db.execute(str)`. if you don't do that, and instead add `db.safe_execute(tpl: Template)`, then you're back to the risk that a user can forget to call the safe function.
also, you're trusting that the library implementer raises a runtime exception if a string a passed where a template is expected. it's not enough to rely on type-checks/linting. and there is probably going to be a temptation to accept `db.execute(sql: Union[str, Template])` because this is non-breaking, and sql without params doesn't need to be templated - so it's breaking some stuff that doesn't need to be broken.
i'm not saying templates aren't a good step forward, just that they're also susceptible to the same problems we have now if not used correctly.
Yeah, you could. I'm just saying that by doing this you're breaking `db.execute` by not allowing it to take it string like it does now. Libraries may not want to add a breaking change for this.
What does db.safe do though? How does it know what is the safe way of escaping at that point of the SQL? It will have no idea whether it’s going inside a string, if it’s in a field name position, denotes a value or a table name.
To illustrate the question further, consider a similar html.safe: f"<a href={html.safe(url)}>{html.safe(desc)</a>" - the two calls to html.safe require completely different escaping, how does it know which to apply?
Some SQL engines support accepting parameters separately so that values get bound to the query once the abstract syntax tree is already built, which is way safer than string escapes shenanigans.
I’d always prefer to use a prepared statement if I can, but sadly that’s also less feasible in the fancy new serverless execution environments where the DB adapter often can’t support them.
For me it just makes it easier to identify as safe, because it might not be obvious at a glance that an interpolated template string is properly sanitised.
> and an extra function can confirm the safety of a value going into a f-string just as well.
Yes, you could require consumers to explicitly sanitize each parameter before it goes into the f-string, or, because it has the structure of what is fixed and what is parameters, it can do all of that for all parameters when it gets a t-string.
The latter is far more reliable, and you can't do it with an f-string because an f-string after creation is just a static string with no information about construction.
> Then the useful part is the extra execute function you have to write
Well, no, the library author writes it. And the library author also gets to detect whether you pass a Template instance as expected, or (erroneously) a string created by whatever formatting method you choose. Having to use `safe(name)` within the f-string loses type information, and risks a greater variety of errors.
Your "old" db.execute (which presumably accepts a regular old string) would not accept a t-string, because it's not a string. In the original example, it's a new db.execute.
Using a t-string in a db.execute which is not compatible with t-strings will result in an error.
Using a t-string in a db-execute which is, should be as safe as using external parameters. And using a non-t-string in that context should (eventually) be rejected.
Yes, but if a function accepts a template (which is a different type of object from a string!), either it is doing sanitization, or it explicitly implemented template support without doing sanitization—hard to do by accident!
The key point here is that a "t-string" isn't a string at all, it's a new kind of literal that's reusing string syntax to create Template objects. That's what makes this new feature fundamentally different from f-strings. Since it's a new type of object, libraries that accept strings will either have to handle it explicitly or raise a TypeError at runtime.
I'm not sure why you think it's harder to use them without sanitization - there is nothing inherent about checking the value in it, it's just a nice use.
You might have implemented the t-string to save the value or log it better or something and not even have thought to check or escape anything and definitely not everything (just how people forget to do that elsewhere).
I really think you're misunderstanding the feature. If a method has a signature like:
class DB:
def execute(query: Template):
...
It would be weird for the implementation to just concatenate everything in the template together into a string without doing any processing of the template parameters. If you wanted an unprocessed string, you would just have the parameter be a string.
I'm not. Again, you might be processing the variable for logging or saving or passing elsewhere as well or many other reasons unrelated to sanitization.
Taking a Template parameter into a database library's `execute` method is a big bright billboard level hint that the method is going to process the template parameters with the intent to make the query safe. The documentation will also describe the behavior.
You're right that the authors of such libraries could choose to do something different with the template parameter. But none of them will, for normal interface design reasons.
A library author could also write an implementation of a `plus` function on a numerical type that takes another numerical type, and return a string with the two numbers concatenated, rather than adding them together.
But nobody will do that, because libraries with extremely surprising behavior like that won't get used by anybody, and library authors don't want to write useless libraries. This is the same.
It's true that in theory `db.execute` could ignore semantics and concatenate together the template and variables to make a string without doing any sanitisation, but isn't the same true of the syntax it was claimed to replace?
Just because templates (or the previous syntax of passing in variables separately) could be used in a way that's equivalent safety-wise to an f-string by a poorly designed library does not mean that they add nothing over an f-string in general - they move the interpolation into db.execute where it can do its own sanitization and, realistically, sqlite3 and other libraries explicitly updated to take these will use it to do proper sanitization.
or other ways of manually interpolating the string - because `db.execute` can flag a `TypeError` if given a string (no matter how it was constructed) rather than a `Template` instance.
Python is not the first one to get this feature. It's been present in JS for some time now, and before that in C# (not sure if that's the origin or they also borrowed it from somewhere). Python adopted it based in part on successful experience in those other languages.
Assuming you also need to format non-values in the SQL (e.g. column names), how does the `execute` function is supposed to make the difference between stuff that should be formatted in the string vs a parametrized value?
However I think using the format spec that way would be dubious and risky, because it makes the sink responsible for whitelisting values, and that means any processing between the source and sink becomes a major risk. It's the same issue as HTML templates providing `raw` output, now you have to know to audit any modification to the upstream values which end there, which is a lot harder to do than when "raw markup" values are reified.
> rather than with some "out-of-band" logic.
It's the opposite, moving it to the format spec is out of band because it's not attached to values, it just says "whatever value is here is safe", which is generally not true.
Unless you use the format spec as a way to signal that a term should use identifier escaping rules rather than value escaping rules (something only the sink knows), and an `Identifier` wrapper remains a way to bypass that.
> Unless you use the format spec as a way to signal that a term should use identifier escaping rules rather than value escaping rules (something only the sink knows)
This should be quiet common in the SQL applications. It will be nice to write t"select {name:id} from {table:id} where age={age}" and be confident that the SQL will be formatted correctly, with interpolations defaulting to (safe) literal values.
The article does mention that the function receiving the template has access to those formatting options for each interpolation, so presumably you could abuse the ones that are available for that purpose?
I think it's just giving an error because a valid AST can't be made, which means valid bytecode can't be made. "<word> <word>" is only valid syntax if one is a reserved word. `nonlocal(foo)` is just fine, of course.
No, it gives an error because `nonlocal foo` requests that the name `foo` be looked up in a closure, but `f` doesn't have such a closure (the `foo` defined outside the function is global instead). `nonlocal` is the same sort of keyword as `global` but for enclosing functions instead of the global namespace; see also https://stackoverflow.com/questions/1261875 .
Here's the statement checking code, which I believe is pre-AST [1]. I would have to dig more to see if that check is there to prevent invalid AST or to just "help the user" (would depend on how they reference the original variable I suppose).
But wow, that's the first time I've seen "nonlocal". In the ~100 packages I have installed, I see 0 usages!
Well, yes, not a lot of people write closures except perhaps when they implement decorators. So there's ordinarily no non-local scope to worry about. People tend to write classes instead, because that's what's familiar.
I think this would have solved the log4j vulnerability, no?
As I understand it, log4j allowed malicious ${} expansion in any string passed to logging functions. So logging user generated code at all would be a security hole.
But Python's t-strings purposely _do not_ expand user code, they only expand the string literal.
Yes and your example is the hero case because it isn't just sugar. A t-string implementation for SQL will of course escape the values which is a common security issue.
Now instead of being explicit all it takes is someone unfamiliar with t strings (which will be almost everyone - still few know about f strings and their formatting capabilities) to use an f instead and you are in for a bad time.
Any sane library will just error when you pass a string to a function that expects a template though. And that library will have types too so your IDE tells you before you get that far.
Such library functions tend to also accept a string as a valid input. E.g. db.execute from the GP usually works with strings to allow non-parametrized SQL queries.
> Such library functions tend to also accept a string as a valid input.
Also? They tend only to accept a string (possibly with some additional arguments, if there is an in-library way to handle parameterization) as input, because Template literally hasn't been an option. New APIs designed with Template available will look different.
A library can extend an existing database library like 'pg' so that PgClient#query() and PgPool#query() require string template statements.
That way 'pg' can continue working with strings, and people who want nice templated strings can use the small extension library, and the small extension library makes it impossible to accidentally pass strings into the query functions.
That is an issue, but essentially it boils down to the existing risk of unknowledgeable people not escaping untrusted inputs. The solution should be more education and better tooling (linters, SAST), and t-strings are likely to help with both.
I suppose lack of overlap in the "interface surface" (attributes, including callables) between `str` and `Template` should nip the kind of issue in the bud -- being passed a `Template` and needing to actually "instantiate" it -- accessing `strings` and `values` attributes on the passed object, will likely fail at runtime when attempted on a string someone passed instead (e.g. confusing a `t`-string with an `f`-string)?
1. Allowing library developers to do whatever they want with {} expansions is a good thing, and will probably spawn some good uses.
2. Generalizing template syntax across a language, so that all libraries solve this problem in the same way, is probably a good thing.