I get the idea (doing fp for a living for 15 years) but the article lives in a bubble populated by mathematically pure unicorns: This strength/feature of fp can only weigh in where we have the mathematical formulas, arriving at those is the hard part in most programming jobs - most of the time those who "define" the product/spec cannot even do it in plain English (because they have to make up things as they go, no criticism, just acknowledgement of bizdev-reality).
I don't find myself scribbling down mathematical equations when doing things like chaining promises in javascript or transforming sequences with LINQ or marking fields as const. I realise functional programming can go in very different directions to things less mundane, but the general principle of avoiding side effects has been a huge win for me when programming in any language, even C++. It's just easier to debug or write something when you can fix some constraints, ie "this method is referentially transparent".
Those are pretty low hanging fruit, but they help a lot.
Yes, please pick those low hanging fruits wherever possible. If you do and I have to maintain your code later I'll send you a bottle of your favorite dram out of gratitude.
But the article (and my criticism) is about the rather high hanging fruit of mathematical purity - of which "side-effect free" programming is just a side-effect (pardon the pun).
Yeah as the other response said, side-effect free programming is great. And while it's a side effect of pure-functional programming, there's absolutely nothing that ties it to a functional approach.
"Lab toy" examples often use simplistic or unrealistic requirements to demonstrate some grand catchy abstraction. But real-world problems often end up being difficult to shoe-horn into the needed patterns. Perhaps the best and brightest can reformulate real-world requirements to fit, but they haven't codified the techniques yet. The last part seems the hardest.
I always envy people who can work on something that can be expressed with math or some other kind of consistent logic. Most company problems are inherently messy and you just end up with a litany of convoluted code.
Example please? Because I'm tempted to respond that there are no inherently messy problems, only insufficient solutions.
What problem could possibly exist that only could be implemented in a convoluted way?
Sounds more like: me and my team made trade offs that were, in retrospect, bad and now we don't know how to transform our code into a new program will fulfills the same requirements but is easier to inspect and understand.
In big companies requirements often change when strategy changes or some top manager has set his mind to something. So you are always behind the curve and you never get a set of stable requirements. Unfortunately that's reality in many companies.
I pity people who have to work with SOAP or Salesforce.
It's not even always about "can we spec this?" or "do we even understand the problem?". But when you get into the real world on a non-isolated system, you have to deal with invalid data, unexpected input, network failures, hardware failures, software bugs, security probing, user errors, memory limitations, timing issues, remote system failures, corrupted data, network failures, data storage failures, network failures, network failures, and network failures.
Did I mention network failures? Also, as beautiful as side-effect-free programming is, any non-trivial program will have side-effects outside the program itself. And your beautiful functional code must account for that.
Hi! I appreciate your honesty.
Of course arriving at the Mathematical formula is the hard part, that's what it means to actually solve the problem.
I consider the fact that you can't code your solution in this style until you have done so as a feature: you can't write code if you haven't really understood and solved the problem. (As opposed to imperative style which let's you implement a half done solution with hidden bugs, not covering all corner cases)
> Of course arriving at the Mathematical formula is the hard part, that's what it means to actually solve the problem.
Most business problems can't be solved with a formula, or with a thousand formulas, or with a million formulas, especially if the problem is “our organization is an illogical clusterfuck” or “we don't even know what we should be doing”.
> I consider the fact that you can't code your solution in this style until you have done so as a feature
It's a feature of programming discipline and intellectual honesty, not of functional programming.
> As opposed to imperative style which (...)
Functional programming doesn't prevent you from implementing “half done solutions with hidden bugs, not covering all corner cases”.
To put it another way - if you don't understand the problem, functional programming helps you be wrong more concisely.
It can be helpful to look for generality and abstraction when solving problems. But it can be unhelpful to assume that general solutions and common abstractions must exist to an extent that makes a given solution more robust and efficient than a patchwork of partial solutions that explicitly handle corner cases.
Understood. Just to be clear: I am saying that in most scenarios nobody understands the problem; people figure it out on the go.
Overall I really like the article because it spells what is an essential feature of fp for me much better than I could have. It is just that it is too far from reality of most programming jobs (unless you happen to be Peter Norvig, working on the Mars rover).
Well, say you're writing code to interface with a mostly undocumented hardware h264 video encoder. Partly through spotty documentation about general use of the video4linux system, partly through that one piece of example code which exists to interface with the particular encoder, you make something which, if given a buffer of a raw I420 video frame, produces a buffer of encoded h264 video frame.
Now, you figure out that the decoder which will decode the video expects the video stream to look different (specifically: the encoder produces only one picture parameter set and sequence parameter set at the beginning of the stream, but the decoder only understands the video if every key frame contains its own copy of the PPS and SPS). After a mix of reading source code and talking to people online, you figure out that there's an (undocumented) way to configure the encoder to include a PPS and SPS with every key frame. You figure that this is really something which must be specified by the h264 specification, so either the decoder is breaking the specification, or the hardware's encoder is; but at the end of the day, it's your fault if the encoded video isn't displayed on the receiver.
Then, you figure out that while you're trying to encode 1920x1080 video, the encoder is only able to encode 16x16 chunks, so the output image is actually 1920x1088. This isn't documented anywhere, but after speaking with the person who wrote the driver for the hardware encoder in IRC, you learn that there's no way to make the driver handle it properly, at least not yet. Therefore, you extract the picture parameter set produced by the encoder, make some changes to have it include information about how the picture should be cropped by the decoder, and splice the modified PPS into the buffer for the encoded frame.
That is what I've been doing at work (with some modifications; the above example is derived from working with two separate hardware video encoders). Even if it was possible to obtain exact documentation on how the encoders worked, sitting down and thinking really hard about how to describe the problem mathematically before writing the code would've been really hard, but because nothing really is documented, I will go so far as to claim it's impossible.
What's the mathematical formula that describes a text box GUI item? Languages that look like math might be great for... mathematical problems. But programs are much more than math.
A good analogy would be music notation (pentagram)
I could describe a piece of music in English prose, telling you were to put your fingers and for how long. The language itself would be easier to understand, but the efficiency of information transmission would be very low.
Chinese looks complicated, unless you know Chinese.
The ultimate criteria to judge a language should be how efficiently (concisely) it expresses the concepts of the subject matter.
At most, I will concede you:
Goodness_of_language = conciseness × parsing_speed_of_an_experienced_user
Understandability is more important than conciseness in a team environment. In a team, you're not writing code for you - you're writing code to be reviewed by other people, integrated with other people's code, and maintained by other people in the future.
In the example from this article, all developers I've ever worked with would understand what the "good old C" example is doing within seconds. Most would understand the Python example. Most of them still working in the industry today would be able to figure out what the functional python code was doing without much excessive effort - it would take them longer, which is bad, but if they were working in it every day I'm willing to believe that they'd parse simple examples like that readily enough.
None of them would understand APL. J is even worse. I stared at the J code for a full minute without any comprehension of the language syntax or what was being accomplished. That code isn't readable or maintainable. It is bad code.
Forget it. It's a different culture, it's void to try to understand this without leaving your previous point of view.
And for the laughs, I've been taught java first, and read a lots of C code, and I had a very bad understanding of what was going on. APL or similar don't cause that confusion. Actually, after years of FP, I now get C a lot more. To quite Hickey, it's not easy but it's simple. There's a set of primitives and operations, a clear system. It fits some peoples brain. For others, imperative state machine are just what they like.
> J is even worse. I stared at the J code for a full minute without any comprehension of the language syntax or what was being accomplished. That code isn't readable or maintainable. It is bad code.
But why does code need to be readable to someone who doesn't know the language? Yes, it's true that Python and some other languages are pretty readable even to people who haven't written any of that language, and that is a nice feature in many contexts... nevertheless, if I'm hiring a Python programmer, I'm either going to hire someone with experience in Python, or I'm going to accept that they're not going to hit the ground running and be massively productive right out of the gate... so how is it really such a loss to write in J and accept that non-J programmers won't understand my code?
Readable code decreases ramp-up time, both in a language and in a codebase. I've never written a line of Java in my life, but if you dropped me in a Java codebase I could start understanding the structure immediately and contributing to it very quickly. If you dropped me in a J codebase I'm not convinced that would ever be the case.
Writing terse readable code is great. Writing code that is so terse that it doesn't have variable names is an antipattern.
Well, I'm definitely aware of what mainstream programming langauges look like. But I think the popularity of a language is only loosely related to its technical attributes.
> The ultimate criteria to judge a language should be how efficiently (concisely) it expresses the concepts of the subject matter.
Some might say that the ultimate criteria to judge a language should be how quickly an inexperienced user starts to be fluent in it.
I studied math (so the syntax isn't really an issue), but I think functional programming is completely inappropriate for general purpose programming tasks (e.g. writing a website, writing an iPhone app, etc.).
> Some might say that the ultimate criteria to judge a language should be how quickly an inexperienced user starts to be fluent in it.
The ultimate criterion to judge a formal language (such as a programming language) is how easily anyone (regardless of expertise level!) can express their ideas in a precise and unambiguous fashion in it.
It isn't clear that this criterion favors any known programming style.
But if you don't know Chinese, it would take quite a compelling argument to convince you to give it a serious try, no? If you're not in China, why not continue in English?
Of course leaning more math isn't nearly as hard as Chinese (for the non-Chinese), but this analogy isn't working in your favor.
I trained as a classical musician and when I picked up the guitar, I was too lazy to learn the notation and learnt the songs by ear and feel instead (this may be why my guitar is collecting dust in a corner). Conversely I know a lot of "amateur" musicians, some pretty advanced, who have learnt songs by ear and can't decipher musical notation of any kind, or understand any music theory.
Chinese looks complicated even to the Chinese, hence Simplified Chinese arriving in the 1950s (and according to Wikipedia, they've started another round of simplification in 2013). Without an alphabet, there are thousands of characters to learn (8,000 in the 2013 version). Ask any Western-born Chinese how fluent they feel and how easily they can read it...
> Too math-like! A step-by-step procedure might be simpler to understand.
A step by step procedure is exactly a mathematical process. Mathematical notation is just a shorthand for some procedures so you don't have to spell out every single step every time. You'd be well served by learning them!
Mathematically equivalence has little to do with readability. Readability is both important and subjective. Sadly, few people investigate what their audience actually finds easy to read.
In the productive world, an easier to understand program is better, even if that implies more code.
Variable names like "TotalAmount" are better than "x".
Also, a 10 line procedural code is better than a one-line function, just because is easier to understand, and easier to change in the right way when the specification change. (it will change 15 times before implementation)
More code in terms of code size in bytes isn't always more complex. For example, sometimes one wants to emphasize the similarities and analogies and spends extra bytes to show that. But that in my experience is rare - and while it could be beneficial for a novice or for a substantially unfamiliar code, for good code if soon becomes burdensome.
TotalAmount instead of x would probably drive crazy most mathematicians. Or most APL programmers. And probably most people demonstrating code on a whiteboard. Even in more mainstream languages like Clojure or Haskell a lot of variables are rather short; it's unfortunate that C++ with history of Hungarian notation, Java or C# have a habit to use long names.
There is a reason why mathematical notation exist. F = gmM / (R^2) is often preferable to "force is proportional to both masses and to inverse of square of the distance with proportionality coefficient gamma".
My advice for you is to try develop a program in REPL fashion. Standard J package from Jsoftware has a good environment for that - you can launch a computation expressed in a line, see the result and if you aren't satisfied just copy the line, make changes and launch it again. Pretty convenient in comparison to classical save-compile-run-checkResults loop of, say, C++ development. One line is certainly easier to change 15 times before implementation when specification change.
It can correlate with complexity positively and negatively.
Which one explanation below is easier to understand?
Photosynthesis? Or 'process used by plants and other organisms to convert light energy into chemical energy that can later be released to fuel the organisms' activities'?
You could argue that Photo+Synthesis might be enough to imply the essential underlying mechanism of the process itself, but I think more people would prefer the latter just for clarity.
I don't think example with photosynthesis suggests anything here. Naming and describing are different things. You can certainly name verbs in J, like functions in other languages, and get the code both shorter and easier to understand, because it would be distracting to always read the long definition. I think more people actually prefer former almost always, except when defining what that the photosynthesis is.
> aren't particularly fond of deciphering complicated math equations?
But it's not complicated. One of the great things about mathematical expressions is that they are simple. But two of the other great things are that they are concise and unambiguous, and it's those two that unfortunately lead many students to think they are complicated.
Mathematical notation is highly ambiguous. See recent discussion of the alleged proof of the ABC conjecture (https://news.ycombinator.com/item?id=15971802 ). If it was really unambiguous then we could parse it like computer code and decide if it is correct or not without all the controversy about what it really means.
Well, mathematical notation is written for humans, not computers. With a bit of context, and comparing to human language (not code), it is very much not ambiguous.
Mathematical notation is not one thing; it is a whole poorly-specified family of context-dependent micro-notations. While I have spent my entire career enthusiastically educating myself on a wide variety of topics beyond anything I was exposed to in school, I have found it essentially impossible to learn any new mathematical notation beyond that which was taught formally. I am capable of understanding novel computer-science concepts perfectly well once they have been translated into a more accessible notation, and of putting those concepts into practice in my own engineering work; but as long as I am struggling through papers which express those concepts in mathematical notation, I understand little and get only as far as I can by inferring what the underlying abstractions must be from the surrounding English text. It very much is complicated, if you aren't already part of an academic system capable of passing it along to you, and so far as my experience has gone there is basically no other way to bootstrap yourself into it. I have spent rather more of my life educating myself outside any academic system than I ever did within one, so the value of a notation with such limited learnability strikes me as being considerably lower than what people who have made a career inside academia might naturally assume.
> it’s usually defined by some of its features, such as: reification of functions, avoidance or banning of functions with side-effects, use of higher-order functions and so on. But, doesn’t that sound like a collection of random features? What’s the motivation behind them?
It's a bummer to pose that question as though there isn't an answer, when the answer is well known and very important.
The motivation of avoiding side effects is because we humans make more mistakes when there are side effects. Functional programming is about making a piece of code not depend on anything outside itself. When you do that, it's easier to get right, and harder to screw up the code. In a word, it's safer.
> I’ll present you a view which, at least to me, makes that set of features seem coherent; and functional programming, like a deep and beautiful paradigm: Functional programming attempts to reduce programs to equations.
We also make more mistakes when we make things too abstract, remove specificity. Write code that is too concise, and you lose meaning, context, and readability.
The "functional python" is the best looking code in the article to my eyes. It explains what it does to the reader, semantically. The APL & J examples aren't very enticing to me. I'm curious about them, but I wouldn't want to work in a codebase where if you don't know what the name of the function means, you can't figure out what it does.
Making code mathier is good occasionally, when working on mathy problems, but generally I'd rather have words and const statements in my functional programs than a concise expression with single letter variables.
"The APL & J examples aren't very enticing to me. I'm curious about them, but I wouldn't want to work in a codebase where if you don't know what the name of the function means, you can't figure out what it does."
Is it possible that APL and J fail this test only because you (and I) don't know how to read APL or J? I think there are some languages that end up being difficult to read due to failings of the language, but I don't know enough to know if APL or J is among them.
Python is, perhaps, more readable than most languages, but I've seen people learning Python have a hard time with many of its choices, so we really have to consider whether we find some version of the code more readable only because we've read a lot more of that kind of code.
Most of the time "easy to use" just means "what I'm used to", and I don't think programming languages are an exception to that.
> Is it possible that APL and J fail this test only because you (and I) don't know how to read APL or J?
Yes and no. :) Its a great point; familiarity and practice is a big part of comfort and productivity, absolutely.
I love playing code golf in Python. You can make Python very concise, and very unreadable. But I don't write professional code that way, and I definitely don't have a goal of making production code as concise as possible. Conciseness is good for very small projects, for the 5 minutes while I'm working in the code. Conciseness is awesome for Project Euler. Too concise, and when I come back to my own code days/weeks/months later, I can't understand it at all. Anyone who's coded for long enough on big projects will know the feeling of coming back to clean, well commented, semantically explicit code years after writing it, only to have no clue what it's doing or how it works.
I also know how to read math, and math (typically... almost always) has this very problem. The conciseness is good for proofs and multi-step algebra, you need it while deriving things and working with the math. Once you codify it into a process, the conciseness is a big obstacle to understanding and code safety. I make lots of mistakes with math because it's too abstract. The mistakes are usually easy to fix because it's only a line or two at a time, but it'd be unworkable and unreadable if I had many thousands of lines of math the same way I have with regular code.
It should be obvious that if a transformation is syntactically trivial, it can only solve trivial problems. If programming is not-math, it's not going to become math just because you change the surface syntax.
A mathematician's definition of a sorted list is different to a programmer's at least in part because a mathematician is interested in a sorted list and a programmer is interested in sorting a list. This difference is intrinsically invariant of the programming paradigm (though not of the context).
This issue isn't so much that this post didn't convince me that there is an underlying, semantically relevant sense in which the APL approach is more mathematical, but that it didn't even seem to notice that this was a thing that needed to be argued.
Programming is _always_ math; was it Dijkstra who said "programming is one of the hardest branches of applied mathematics"? It's also engineering - as in "engineers have to do some math occasionally, even if they don't realize that".
I often ask on the interviews to produce an expression returning sorted 3-element list using some simple primitives. I've found that with chosen primitives you still can model different approaches - and even though in all cases the result is an expression ("sorted list is..."), the structure of that expression can model imperative style ("first do this, then this...")
On one hand, yes. On the other, classifying whether programming is a math or not doesn't help you decide whether you should use APL, or much else for that matter.
I wrote my own functional, Haskell-like, language, and after a couple of years and seeing a couple more code bases in various languages, I am now starting to firmly believe that currying should be strictly avoided (or at least highly discouraged). Obscure symbol operators with operator precedence are also a clear readability problem.
But unlike what this article suggests, these traits are not essential to functional programming. As long as you stick to functions which given values return other values, and your language has support for passing functions to other functions as well, you can reap most of the benefits of functional programming.
Most of us spend 90% of our time reading other people’s code. So even though writing a concise and elegant formula in J might be a satisfying puzzle, I personally don’t want to deal with the cognitive burden of point-free style when sifting through my colleagues’ pull requests.
you might know that it’s usually defined by some of its features, such as: reification of functions, avoidance or banning of functions with side-effects, use of higher-order functions and so on
The "function" in "functional programming" and the "function" in "avoidance or banning of functions with side-effects" are two different words, by the way! FP is named after the math word, not the programming language word, which means different things in different languages but usually means a kind of subroutine. FP means programming in a style that emphasizes composition of mathematical functions, and in an imperative language this generally amounts to writing your subroutines in a certain way. It just so happens that many imperative languages call subroutines "functions."
because adverb (/ in this case) binds with + before three verbs +/ , % and # are grouped into the fork. So some parentheses aren't necessary. It would be cleaner to show three expressions which share the common part as similar in letters as well.
Logic programming is close to this. In functional programming, left side of equation always consists of dependent variable and nothing more. Logic programming allows to put any variables on both sides of equation.
This is an interesting post, and the code sample from APL got my attention. I have questions.
How do I learn/get/run APL or J? How do I even type the APL code given? What does the J code even mean?
As for the content...
"makes code super concise (less room for bugs)"
Unfortunately, while this sounds great, it does not logically follow. And while geometric mean is well-defined and understood, most functions I'm probably going to write in whichever language are going to be more complicated and less well-defined. So, how does APL/J/pure-functional help/make my non-trivial code more beautiful/concise/bug-free/provable?
They are unicode characters. Type them the way you would type any unicode character. If you have not yet found a practical way of doing it, there are plugins for editors to help you type APL characters with no hassle.
>How to learn APL?
There's a Dyalog APL manual which is quite clear, Google it. Also, you can use the GNU APL interpreter.
I would recommend learning APL first, only by reading the Dyalog manual and writing programs with pencil an paper, then learn J in detail.
> How do I learn J?
Jsoftware provides a free interpreter. They also have a sleek interpreter app for Android. I would recommend to download this app and go through the project Euler problem set. Jsoftware provides all the documentation you will need in their webpage
> The best would be to learn it and see it for yourself.
I've found the statement on the front page of jsoftware.com to be rather accurate:
"If you are interested in programming solutions to challenging data processing problems, then the time you invest in learning J will be well spent."
And "data processing problems" could be understood pretty widely. I've had a case when I first spent about 45 minutes producing a prototype (a program which would generate realistic plans for floor of an office building) using J, then converted that prototype in functionally equivalent (no embellishments) program in C# spending a couple of hours. Certainly working on a prototype took several failed attempts, yet it was still faster than doing the same in C# after the algorithm was already clear.
One could also say that functional programming prefers expressions to statements —and treats functions as just another expression. When needed, statement-like things are constructed from expressions.
Equations in lambda calculus do seem to have a "preferred direction" specified by the evaluation rules, though referential transparency lets you go "in the other direction" in your head, when reasoning about your program.
Perhaps the programs ~ equations idea would apply even better to logic programming, or term rewriting?
This is exactly it. I don't write 100% functional code, but when given a possibility of implementing functionality using state vs pure function, I aim for the pure function whenever possible.
I am something like an intermediate programmer and this is useful.
I want to understand how FP replaces OOP in a similarly concise example. I have a sense that it involves a reconsideration of why OOP is useful, and solving that issue with a widely new approach, but I am not there yet myself. I’d love to see something like these Python comparisons but involving the demolition of an OOP implementation.
Thanks. OK. This gave me a good outline. These ideas were somewhat right under my nose, in some of the Python concepts like closures and decorators. I don't know the languages here quite the same but the possibilities are convincing nonetheless. Hmmm... I could see myself getting carried away with this if I put some time into it.
I think the main feature of Functional programming is not avoiding side effects but abstraction. You can compose different kinds of powerful abstractions and programmers are free to choose their "language" of abstractions, compared to "X Oriented" languages that forces programmer to think in certain terms and limits in ways you can create layers of abstractions.
In the author's example of the geometric mean, shouldn't the function return an error if any of the numbers were negative? Because the program shouldn't be sending negative numbers to that function to begin with so he/she should maybe get some hint that one of those functions producing said numbers isn't working properly. And if there are no positives, I sure as hell don't want a divide by zero error bringing the program to a complete stop when I could just simply display "Could not calculate geometric mean, check the inputs" to the user.
The functions in the article are not the geometric_mean. They are functions which first filter the list and then take the geometric mean. I only misnamed them that way for the sake of exposition. The only true geometric_mean in there is the J version
This quote made me smile.. Thanks OP! Happy holidays!
> Just in case all that wasn’t enough to convince you about the superiority of the functional gospel, take this threat: FP will displace OOP, so start thinking along our lines our you will eventually lose your job
"The fact that the
functions are collected into records and called methods is
irrelevant. As a result, the typical object-oriented program
makes far more use of higher-order values than many func-
tional programs” - William Cook, On understanding data abstraction revisited.
OOP, especially with a focus on immutability, is not that far away from FP. It's not like, say physics where two domains can be conceptually quite different. OOP and FP are just two programming tools meaning that their variation is quite limited by the domain they are in.
Objects as a higher-order abstraction, as an encapsulation of data and methods, are all fine and dandy. When I think of OOP, however, I think of deep inheritance. I'm not a great fan of the latter.
