Nonsense. We've figured out how to do type systems. We can even fully infer types if you're willing to accept some quite reasonable restrictions on how polymorphic your code is. We have a bunch of reasonable approaches to effect tracking, which Lisp folk used to have to do by hand (that story about the T garbage collector sounds like the most unmaintainable piece of code I've ever heard of). We know how to solve the expression problem.
Modern "pragmatic"-oriented languages have little that wasn't in Lisp - or in Algol. Which is fine! The innovation is about the wider ecosystem (a lisp programmer from the '90s would be amazed by CPAN, never mind maven), or about making common tasks easy (a lisp programmer might eventually put together a collection of idiosyncratic macros with much of the functionality of Rails, but that wouldn't be a project you could hire someone else to maintain). OTOH modern "researchey" languages, like Coq or even Haskell, are far far ahead of Lisp.
> OTOH modern "researchey" languages, like Coq or even Haskell, are far far ahead of Lisp.
I know Common Lisp well, and I honestly tried to learn and to use Haskell. I realized that Haskell has the advantage of a strong type system but it seems only to be useful for language research (compiler writing) and mathematical applications. Haskell is (in my case) almost useless for every day real world applications. It is a very good choice if I know the algorithm and data structures exactly in advance. But real world applications often require changes which are not predictable. It is a pain to align a whole Haskell project according to new requirements to make the whole system work again. It takes far too much time, and I wonder where the benefits are if other weaker typed languages can do the same job as well, or even better.
In the Lisp world I can develop prototypes very quickly. And if I need a strong type system as in Haskell then I can use Shen (shenlanguage.org) which has a strong rule based type system on top of Lisp. That means that I can use both worlds -- the default dynamic system of Lisp, and the strong type system of Shen.
Lisp is still very powerful and yet small in comparison to the huge LLVM system of Haskell. It can be used on embedded systems (ECL for instance) which Haskell is not able to do (AFAIK).
I don't know what you exactly mean with "tracking". If that means tracing then Lisp even offers backtracking (debugging back in time), and I think almost every Lisp programmer knows Slime.
So the claim that Haskell is "far far ahead" of Lisp is not true at all. The only real weak point of Lisp is a good GUI library. I wish there were strong support of Qt and HTML5.
Another favorite language of me is Nimrod because it joins the best of several languages (Python, Perl, Lisp and C) together.
> It is a pain to align a whole Haskell project according to new requirements to make the whole system work again.
I'll quote a recent tweet by Chris Done: "I feel like 80% of Haskell advocacy should involve screencasts of people refactoring large codebases."
In my experience, refactoring is easier with a strongly typed compiler, not harder. It may take more time and work to get your program to "run" again, but the end result will be more likely correct, and more likely to be a better design.
This has generally been my experience as well when I'm working with almost any reasonable type system. Every time I've focussed on leaning on the type system, it's made refactoring much quicker for me and left me feeling way more confident.
Every time I encounter a bug I think "is there a way I could have encoded this into the type system so it would have never compiled with the bug?" The thing I like about Haskell is how often I can say "yes" to that question and implement it. From what I understand, that question would would "yes" even more often with dependent types, though I've yet to really learn much about them.
I really dig lisp (especially clojure) but the thing I miss when I'm using lisp is a really solid type system.
There are other ways of making refactoring easy. I'm not trying to dismiss a powerful type system - which is a great help indeed - but pervasive unit tests and contracts work as well. Linters are a nice addition, and even gradual typing like in Erlang with Dialyzer is much better than nothing. The point being: problems solved by type systems are being solved with other tools, too. I don't want to argue about how well these tools work, because all of them (including static type systems) have relative strengths and weaknesses; it's just unfair to present refactorings or correctness guarantees as benefits reserved for type systems alone.
I'd argue that you can rely more on powerful types because they lend more to the totality of your program. This is just a hunch/intuition though. If I'm wrong, I'd be glad to know why and how so I can correct that ;)
I know CL, I know Haskell. Really what OP said has nothing to do with refactoring (which is indeed painless in Haskell) but more to do with interactive programming.
Haskell is not terrible at interactive programming, but compared to Common Lisp and Smalltalk, it's garbage.
And this is exactly why it feels so kludgy and awkward when
one tries to do bottom-up programming and organically evolve his program, the standard way of programming in CL and Smalltalk since they are image-based languages that are designed for exactly this.
This is one of the benefits that one with no prior experience in working in this way can quickly attempt to dismiss, but it's really THE most important reason for someone to use CL rather than Haskell. By having a language that allows you to program in a live environment, from the inside and keep the entire system running at all times, you become one with the machine. The feedback loop is kept extremely short, and you're in a constant state of flow.
It's too bad when people dismiss CL (and Smalltalk) based on trivialities (type systems) that are completely irrelevant in the grand scheme of things. Writing code in Haskell to me feels like a sterile, boring process similar to working out mathematical proofs. That's not to say that Haskell doesn't have its place. When the domain is well-defined and I know exactly what I want to do and how I'm about to do it, I'm more inclined to reach for Haskell or Ocaml than Common Lisp.
This is not the case for me most of the time however. When I program for myself, I find that I often have a general idea of what I want to do but no specifics in mind. It is then that I reach for Common Lisp in order to flesh things out and
get inspired if you like. Or, similarly, when I'm doing a project that is hard, the domain uncertain, something with solutions that are not obvious ahead of time. Common Lisp and Smalltalk shine for these sort of problems, there is nothing else that comes close in my opinion.
Common Lisp but not Scheme/Racket I hear you say? Yes, this is indeed the case.
Racket is just bad at interactive development of the kind I just described, because its developers deemed image-based programming 'too confusing' and removed all support for it
from the IDE. This should also explain why it's not mentioned at all in the original article. Moreover, I find that in order for someone to truly appreciate how empowering image-based development is, one needs years of writing code in traditional non-image based languages. Which is why the best feature of Lisp in my opinion is swept under the rug by most newbie programmers who try out Lisp. They lack the experience to realize what they are missing.
Which is also why the original post author completely misses the point. The real power of Lisp comes from what I just described, not from 10 or 20 bullet points that one needs to side-by-side compare with other languages. By his admission he is a newbie programmer, and a newbie to Lisp too, made evident by his misunderstandings about the Common Lisp macro system vs Scheme "hygienic" macros. Hopefully he will grow and learn to appreciate Lisp for what it really is.
Finally, writing code in Common Lisp or Smalltalk makes me feel like an architect of my own private universe. This is exactly what programming should feel like, an organic process, fusing mind and machine together if you like.
If it takes longer to refactor the program, that can kill the initiative to do so.
When you do get the program to run, but it turns out that the refactoring is not wanted for whatever external reason, then you just wasted more time than was necessary.
Without the bondage of static typing, you can just refactor a small enough fraction of the program, which can run now in some limited way. You can use that to motivate yourself, or to pitch the idea to other people.
It is helpful to be able to play "what if" without committing a lot of unnecessary time and effort.
I fully understand the idea about motivation (and my "let's muddle through"-oriented brain is always tempted by this), but I think it goes both ways:
If it takes short to refactor the program but, because of the lack of safety checks, what seemed to be working "in some limited way" turns out to be a big ball of bug-ridden mud in the end, and maybe the whole approach wasn't sound, wouldn't that also kill motivation for future refactors? And what if keeping the motivation high for the more impulsive programmers leads to worse software?
Doesn't this look like something analogous to the old carpenter's motto: measure twice and cut once? "But I don't want to measure, I want to cut it now, it'll be faster! Measuring kills my motivation!" :)
> It is a very good choice if I know the algorithm and data structures exactly in advance.
I'd say the exact opposite: the great strength of Haskell lies in its powerful support for abstracting things out so that these things can easily evolve independently. The hard part of taking advantage of this seems to me to be that we're used as programmers to not having those tools, so it is hard to get used to thinking about doing that kind of abstraction.
I understand that. I really like Haskell but I wonder if the type system is actually too strong for the real world.
Generalizing algorithms sounds nice but if Haskell is so powerful, why is the Haskell community still not able to provide a convenient working package manager? Perl has CPAN, Ruby has Gems, Lisp has ASDF and Quicklisp but Haskell is still stuck with buggy Cabal.
I have been in the Cabal hell many times. I know that there are sandboxes, and I considered to install Nix and NixOS just to use Haskell. It surprises me very much that a simple package manager is such a big issue in Haskell. This is a typical example for a real world application that causes strange difficulties in Haskell where other languages have no issues at all.
FPcomplete is already working on a new solution with snapshots. I am curious how convenient it will be.
Cabal is not buggy. Or, more precisely, Cabal Hell is not the result of bugs in cabal. It is the result of 1) static linking with cross-module inlining, 2) a lot of diamond dependencies, and 3) a lot of breaking changes in the ecosystem.
1 is arguably an artifact of Haskell - performance really suffers if you can't do this - but also something that's arguably desirable generally.
2 is the result of the ability to provide abstractions and tools that are useful in a phenomenal number of contexts (mostly a good thing) and the failure of the language-as-standardized to provide these directly (mostly a bad thing).
3 is the consequence of an active community that values experimentation, is willing to try new things, and places an emphasis on Getting It Right. This is actually more mixed a blessing than it sounds.
> I really like Haskell but I wonder if the type system is actually too strong for the real world.
Given the amount of "real world" programming that's been done with Haskell, I suspect not.
> Generalizing algorithms sounds nice but if Haskell is so powerful, why is the Haskell community still not able to provide a convenient working package manager?
I don't see a real problem with Cabal with sandboxes.
> Perl has CPAN, Ruby has Gems, Lisp has ASDF and Quicklisp but Haskell is still stuck with buggy Cabal.
Cabal doesn't seem to be buggy, and is about on the same level as Ruby Gems (I think the difference is that the version bounds conflicts that manifest at package install time in cabal occur at runtime with gems, because rubygems installs multiple versions in the local repository but needs to import a single version at runtime (bundler, which is separate from gems, addresses this, among other practical issues with gems.)
Dependency hell is common to package systems that don't have a single version of every package that is curated to work with all other packages in the set -- CPAN isn't immune to it either, at least from what I've seen on the web (I haven't used Perl at all in many years, and only very little then.)
Cabal now that is has sandboxes available, seems at least on par with gems.
> FPcomplete is already working on a new solution with snapshots.
Its really an intermediate stage between what package repositories/managers like cabal/gems/cpan provide and what Haskell Platform provides. Its not really a "new solution" for the same issue. [1]
Wait, are you suggesting that the Haskell type system is too strong to write a package manager? That makes no sense. A much more valid reason that Cabal is annoying is that Haskell is a compiled language that uses static linking, unlike all the languages you mentioned, which are interpreted.
SBCL also uses static linking. It doesn't recompile every ASDF imported library whenever I compile my application. It just does so when it is actually necessary. Why doesn't it work so easily in Haskell?
I think it's the strong type system because every small code change can break the whole fragile structure of a Haskell application. It's like wheels in an old clock. If you break one jag of a gear then the whole system stops. You have to take apart all gears, change the affected one, and assemble everything together. Cabal hell breaks loose when you realize that someone changed the shape of a small gear, and you don't have access to the old one, except you stored some in your sandbox. Over time you get a whole farm of sandboxes full of obsolete parts. Then after a long time when you have to maintain some of the old code you realize that you have to edit even the foreign (!) libraries yourself because they are totally obsolete and incompatible to the current ones :-) Could that be the reason why there are (AFAIK) so many hackage projects aged or even unmaintained?
> SBCL also uses static linking. It doesn't recompile every ASDF imported library whenever I compile my application. It just does so when it is actually necessary. Why doesn't it work so easily in Haskell?
It does; in fact, in GHC -- the dominant Haskell implementation -- at least, its even easier: it doesn't recompile installed packages period. Packages are precompiled, and are merely linked when you compile an application that they depend on.
Aside from it being factually incorrect, I'm not sure why this is even relevant to the discussion here -- "dependency hell" doesn't come from recompiling libraries when you recompile applications, it comes from conflicts between version requirements for dependencies among packages installed in a single repository (which is why sandboxing is a tool for addressing it.) It manifests when you try to install a package with dependency conflicts with another app, not when you try to compile an app that depends on the conflicting packages.
> I think it's the strong type system because every small code change can break the whole fragile structure of a Haskell application.
I think before you try to explain why something happens, you should first be sure that the thing you are describing actually happens.
> Could that be the reason why there are (AFAIK) so many hackage projects aged or even unmaintained?
I suspect the reason is the same that there are so many Ruby Gems in the main repos that are aged or even unmaintained (and I'd be surprised if this was different in any similar package database that wasn't actively culled of non-current projects -- which would have its own downside) -- the repository system is, by design, low cost to enter, so lots of things that have no strong demand or lasting commitment get posted to it.
Disagree about Haskell being even remotely useless for every day real world applications. Haskell is good for reducing complexity and being able to rely on guarantees. A couple months ago I wrote a screen scraper/web interface that was basically an expired domain purchasing application in Haskell.
> a lisp programmer from the '90s would be amazed by CPAN, never mind maven
CPAN was started in 1997. A lot of Lisp programmers back then were also doing Perl, and a few people who were doing Perl then are doing Lisp now.
Whoever is "amazed" by Maven should go jump off a bridge.
> We have a bunch of reasonable approaches to effect tracking, which Lisp folk used to have to do by hand
I'm assuming you're talking about generalized arrows or something and not monads. Generalized arrows partly came out of work on linear types and parallelism done by Lisp people in the 80s and early 90s.
Rails doesn't have functionality, Rails has a bunch of shitty complicated ways of doing web programming, with a bunch of shitty poorly maintained Ruby libraries (most of the good ones are just FFI into C libraries).
If you're going to talk about modern "practical" languages, you could do better. Go is pretty great except it doesn't have exceptions. JavaScript solved code distribution and program-level virtualization. Those are much better comparisons.
> Nonsense. We've figured out how to do type systems. We can even fully infer types if you're willing to accept some quite reasonable restrictions on how polymorphic your code is.
To be clear, they were far ahead of lisp in the 90s too.
The major innovation is probably more societal in these languages. We discovered that they really are much better for writing reusable abstractions on the level of the whole community. We also ran into some issues with scaling those abstractions at the level of 10s of thousands of packages and are slowly coming to solve them.
Type systems are the one exception, but it still remains broadly true that Lisp was way ahead of everybody. There are many things we could talk about besides type systems:
Lambda expressions - just now reaching Java and C++, been in Lisp forever
Garbage collection – (obviously)
Turing-complete (edit: fully evaluating) macro systems – I've heard C++ is moving in this direction (not sure to be honest) but Lisp is still ahead on this
Gradual/optional typing – Others have been moving towards this, CL has had it forever
Interactivity/REPL – e.g. Swift may finally give a REPL to systems programmers, Lisp has had it forever
Dynamism/late-binding – Lisp is still way ahead on this, goes hand-in-hand with interactivity
Everything serializable – Lisp is still ahead
Structured code editing (paredit) – Lisp is still ahead, not sure anyone else even knows what this is
Multiple inheritance – I think this is coming to Java finally
Image systems (hibernating a running process) – Still ahead
Condition system / advanced exception handling – Still ahead
Functional programming techniques (map, reduce, etc.) – Still ahead
Keyword arguments – Recently added to Ruby, Lisp has had them forever
I don't know about Swift but equating a REPL to interactivity is a horrible disservice to Lisp (and Smalltalk).
For example, Python has a REPL but it is not very well suited for interactive programming. For example Python does not handle you redefining a module after you import it without some voodoo. Nor it has forward references. For example in Lisp I can define a method that especialices on a class I haven't yet defined. Or define a function that calls function B which I will define later in the code. Nor have I seen something akin the slime Inspector and presentations, which are not quite there as the Symbolics Genera environment for interactive programming
Excellent point. I found myself struggling to describe Lisp in terms that non-Lispers would understand (blub paradox I guess), and I figured almost everyone knows what a REPL is. You're absolutely right that a REPL itself is just scratching the surface, and there is all kinds of interactivity beyond that.
Still ahead of mainstream programming languages, which are typically imperative by design, and usually overly "noun" focused compared to the "verb"-friendly nature of Lisp. Adding map/reduce/etc. to the noun languages is a bolt-on that is not exactly ideal.
Take Java 8 which now has map: sure, you can pass a lambda expression with a custom function body, but is using functions (particularly side-effect-free ones) as arguments idiomatic otherwise? I can only imagine the amount of ceremony code that ensues when trying to merge those worlds. Don't get me wrong, having map/reduce/etc. is an improvement to Java for sure, but Lisp is still well ahead on the overall cohesion and utility of such things within the context of the rest of the language.
Multiple inheritance is much more complicated and restricted in C++ than in Common Lisp. As with many things in C++, you really have to know what you're doing to use it correctly. In CLOS it pretty much just does what you expect, and only when you want to do something unusual do you have to learn the finer points.
"Turing-complete macro system - I've heard C++ is moving in this direction (not sure to be honest) but Lisp is still ahead on this"
1) Turing-complete is a bug, not a feature.
2) C++ template metaprogramming has been Turing Complete for ages.
3) There are nonetheless significant limitations on what you can do with templates in C++, compared to what you can do with macros in Lisp.
"Turing Complete" is a theoretical construct that doesn't really mean much other than giving you easy ways to prove that certain questions about the system can't always be answered. One significant reason for this is that "Turing Complete" merely means that you can embed any computation in the system in some encoding - with something like a macro system, the encoding you generate is the whole point.
I understand what you're saying. What I mean is not the trivial "nand is turing complete" sense, which I agree is harmful and silly, but full eval capabilities. Lisp macros can fully evaluate Lisp code, and I consider this a great feature, but I don't know how else to describe them. "Full evaluation macros"?
To my mind, having eval available is a cheap way to make sure you are sufficiently powerful. In principle, it's not the best way. In practice, it's arguably the best way out of options at hand, though some alternatives compete.
> What I mean is not the trivial "nand is turing complete" sense, which I agree is harmful and silly, but full eval capabilities.
The two are equivalent (provided that the target language that can be evaluated is itself turing complete) -- that's rather the nature of turing completeness. If you have the former, you can make the latter with it and nothing else; if you have the latter, you necessarily also have the former.
Yes, it was a poor choice of terms. To be clear, eval isn't equivalent except within the narrow theoretical notion of turing completeness: it can do I/O, for instance file I/O. And it accepts high-level language code directly which is very important in a practical sense compared to, say, accepting nand instructions only.
It's hard to describe because even "full access to the language at compile time" would be useless if that language were C++. The advantage of macros is that you have access specifically to Lisp at compile time, not just any old blub language. And that just circles back to "Lisp is great". :)
Newer features tend to use type systems for part of their implementation because they're a powerful tool, but I feel the feature is its own thing. I'm excited about typeclasses, or anything else that solves the same problem. I'm excited about effect systems, which tend to be implemented as types but don't have to be. I hope to learn more about the matryoshka approach, which seems like the opposite thing or at least opposite emphasis to lisp's approach to metaprogramming (which emphasises doing the metaprogramming in the same language as the program itself).
>a lisp programmer might eventually put together a collection of idiosyncratic macros with much of the functionality of Rails, but that wouldn't be a project you could hire someone else to maintain
Common Lisp has about 10 different anything frameworks, which nobody in particular uses in favor of writing their own, mostly because they can. Many of the ten are in fact written into the standard.
Common Lisp isn't so much a "language for writing languages" as a giant bag of everything.
to quote an old quip: you have a problem. common lisp has a clunky solution that's built into the standard, and scheme has a beautiful one that doesn't work in your dialect.
We can even fully infer types if you're willing to accept some quite reasonable restrictions on how polymorphic your code is.
Personally, I don't consider "don't pass a polymorphic function to another polymorphic function" and "don't demand a polymorphic function as an argument" to be all that reasonable.
But you only have to infer types because you insisted on removing them from the run-time information.
Sure, you're clearing the road of fallen logs, but they fell there by your own axe.
Having the type there at run time is smarter than working up a lather prior to run time to avoid having it there.
Also, real systems don't have a well-defined "compile" and "run time": the two are intertwined. Compile time is just the compiler's run time, and we need access to the language at that time, and has to be type-safe also, and we would like that comile-time code to itself be compiled (and to have access to the language during that code's compilation ...).
Modern "pragmatic"-oriented languages have little that wasn't in Lisp - or in Algol. Which is fine! The innovation is about the wider ecosystem (a lisp programmer from the '90s would be amazed by CPAN, never mind maven), or about making common tasks easy (a lisp programmer might eventually put together a collection of idiosyncratic macros with much of the functionality of Rails, but that wouldn't be a project you could hire someone else to maintain). OTOH modern "researchey" languages, like Coq or even Haskell, are far far ahead of Lisp.