I think more analogous to the examples Kay was giving would be for a single program to quietly change the rules for function application, or to quietly change the behavior of CLOS.
I think of an embedded DSL, based on a macro like it is here, as a bit different than that, because it's not doing anything quietly -- it applies only to the parenthesized extent of the code that begins with your macro name:
(my-dsl-macro my dsl fills the rest of the parentheses)
People who don't already know what `my-dsl-macro` is see the documentation or code that says it's a macro. Even were it a normal function, people reading the code should probably know what the function does, so maybe they have to glance at where the IDE automatically displayed the syntax or documentation for them, anyway.
Ideally, your editor will also color the macro names differently, especially for readers who don't know what's a macro or a function. (Rust adds an exclamation mark to the macro name for a use, which is a good idea when you don't already know what's macros, but maybe a bit annoying to have all those exclamation marks when you do know that, say, `println` is a macro.) But even if you don't know the name is a macro, you'll be clued in if the text within it doesn't look like your top-level language, which it often doesn't (e.g., SQL in s-expressions doesn't look like base CL).
A key is to use DSLs judiciously -- for improved readability (for your base language programmer, or domain experts), maintainability, and/or performance. Maybe not for, say, a convenience for the sake of minor code terseness improvement only.
Of course, SQL is a whopper of a language, and perhaps overkill if you invented it as a DSL for this particular application. (More suspicious would be to invent your own relational query language that seems gratuitously different than SQL, when SQL already existed.)
As for changes more like I think Kay was talking about, in the Racket (Lisp family) universe, outside of a macro, normally you wouldn't do that, but when you have a good reason -- say, you want to prototype a lazy language, or implement a specialized language for a GPU programming backend, or a DSL for your domain experts -- you can. In that case, you'd have a `#lang` line at the very top of the file that tells you what different language this is, instead of `#lang racket`. You might make that produce modules that can interoperate with modules of other `#lang`s, but you're not doing anything sneaky that breaks the language of those other modules.
You still have the problem that the `my-dsl-macro` is a black box.
That macro can do anything it wants to the sub-AST. The problem is not about hygiene re: variables but about the meaning of symbols. A function application within it that doesn't obviously have anything to do with the DSL is also subject to the macro's will to have its meaning changed. This can make composition difficult. Who guarantees that if you combine `my-dsl-1-macro` with `my-dsl-2-macro` or simply with general purpose code, that they don't interfere in odd ways?
Granted, this is not a big concern for a well-designed and widely used macro. However, the bottom line is that the inherent freedom of the abstraction allows for issues to arise whose debugging require a deep understanding of the involved macros and their implementations.
You don't really have the same problem if there is only procedural abstraction. The boundaries are clearer there. Even if you do something like the Interpreter Pattern or Haskell style AST-constructing EDSLs, you have guarantees that whatever AST is constructed cannot inspect whatever non-DSL code you combined with it.
> Who guarantees that if you combine `my-dsl-1-macro` with `my-dsl-2-macro` or simply with general purpose code, that they don't interfere in odd ways?
Nobody does. But that's where the power lies. You can't have unrestricted power with restricted features. Lisp gives you unrestricted power. It's up to you to use it correctly, for w/e definition of correct your context makes sense.
Yes, but the argument was rather along the lines of "here's why I think (some people think) we shouldn't have unrestricted features" or "I don't want ... because...". Languages don't necessarily need macros (many don't have them) so arguing on the basis that the dangers of macros are an inevitable cost for having them is correct but it works under the assumption that macros are indeed desired by everybody.
It's a bit like arguing about pointers and memory safety.
I do think there is a lot of value to look into these things and research alternatives to macros. For instance the problem I have described could be addressed by a new macro-like feature that has to respect certain boundaries.
True, `my-dsl-1-macro` potentially does anything inside its syntactic parentheses extent, including possibility manipulating syntax within it that someone just glancing at the code might assume is a code snippet that's pasted in verbatim, including what looks like a use of `my-dsl-2-macro`.
The verbatim paste seems like the usual case for macros.
This kind of potential expectation violation isn't specific to macros. For example, some functions could potentially mutate arguments when, at a casual glance, the expectation would be that that it doesn't mutate. (Even with an explicitly const argument, the language usually permits mutating something referenced by that immediate const, so we're back to expectations again.)
I think this goes back to conventions and judiciousness. And perhaps also IDE-supported easily-accessible documentation, and good naming.
Sure, the point I was trying to make is that macros add yet another layer of potential violation of expectation. It's hard (for me) to say whether their dangers outweigh their benefits (my guess is probably not), but it is something worthy of consideration to understand where anti-macro opinions come from.
That concern sounds reasonable. It doesn't help that there's wide variation in the nature and quality of macro/DSL/minilanguage/metaprogramming stuff people see.
Personally, I'm very comfortable working with reasonable programmers using macros in Racket. And most CL programmers are very sharp, and could be trusted to use restraint with their more dynamic tools (especially if you're talking about work, rather than creative personal side projects).
For that matter, some non-macro/template overriding features of C++ pack more astonishment than syntax extension in Racket. And I wonder how much some suspicion of a good macro system is due to prior experience with languages like C++.
I've never wrote commonlisp for work, but what I've read from people doing so is that they're not fools. All those I've seen had a clear notion of when and how much to abuse lisp power. Ugly is ugly and rare are seasoned lispers that are still confused on this.
I think of an embedded DSL, based on a macro like it is here, as a bit different than that, because it's not doing anything quietly -- it applies only to the parenthesized extent of the code that begins with your macro name:
People who don't already know what `my-dsl-macro` is see the documentation or code that says it's a macro. Even were it a normal function, people reading the code should probably know what the function does, so maybe they have to glance at where the IDE automatically displayed the syntax or documentation for them, anyway.Ideally, your editor will also color the macro names differently, especially for readers who don't know what's a macro or a function. (Rust adds an exclamation mark to the macro name for a use, which is a good idea when you don't already know what's macros, but maybe a bit annoying to have all those exclamation marks when you do know that, say, `println` is a macro.) But even if you don't know the name is a macro, you'll be clued in if the text within it doesn't look like your top-level language, which it often doesn't (e.g., SQL in s-expressions doesn't look like base CL).
A key is to use DSLs judiciously -- for improved readability (for your base language programmer, or domain experts), maintainability, and/or performance. Maybe not for, say, a convenience for the sake of minor code terseness improvement only.
Of course, SQL is a whopper of a language, and perhaps overkill if you invented it as a DSL for this particular application. (More suspicious would be to invent your own relational query language that seems gratuitously different than SQL, when SQL already existed.)
As for changes more like I think Kay was talking about, in the Racket (Lisp family) universe, outside of a macro, normally you wouldn't do that, but when you have a good reason -- say, you want to prototype a lazy language, or implement a specialized language for a GPU programming backend, or a DSL for your domain experts -- you can. In that case, you'd have a `#lang` line at the very top of the file that tells you what different language this is, instead of `#lang racket`. You might make that produce modules that can interoperate with modules of other `#lang`s, but you're not doing anything sneaky that breaks the language of those other modules.