> Putting too many functions that are relatively complex in the where clause is a bad idea, because you lose the explicit type signature…
Note that it’s possible to give type signatures for definitions in “where” clauses:
twice :: Int -> Int
twice x = two * x
where
two :: Int
two = 2
However, you may need the ScopedTypeVariables extension[1] in order to be able to express the correct type signature. (This extension will probably be standardised at some point.)
The biggest real problem I’ve found with large “where” clauses is that they tend to have a lot of implicit dependencies on variables from the enclosing scope. Sometimes you want that for convenience, readability, or performance, but lifting local definitions out into top-level definitions can also help make them more explicit and reusable.
>The biggest real problem I’ve found with large “where” clauses is that they tend to have a lot of implicit dependencies on variables from the enclosing scope.
For me it's always a toss up. I like to make things local where possible to give some indication that the definitions need not be looked over with a fine-toothed comb -- they aren't used extensively. On the other hand, it does often lead to implicit dependencies. It would be nice to have scope highlighting - color every identifier based on the distance from the scope it's defined in. This idea shamelessly stolen from Doug Crockford in https://www.youtube.com/watch?v=b0EF0VTs9Dc
Dr Racket has a nice feature where it draws arrows between a variable and its binding site, and between a binding and its use sites; it would be nice to use a similar thing in Haskell.
> Putting too many functions that are relatively complex in the where clause is a bad idea, because you lose the explicit type signature (you should always specify it for top-level functions).
You can give functions in the where clause type signatures. It isn't as common, but it might be preferable to breaking out code into separate functions, depending on how well the function makes sense outside the context of its "parent" function.
Great post, I remember when I first picked up a functional language (it was OCaml for me) I was perplexed by the sheer number of concepts that were all new to me. Currying, tail recursion, higher order functions, combinators, functors, lenses, the list goes on and on. These concepts can be translated outside of the functional realm to a certain extent but the way that many of these functional languages embrace these ideas as part of the paradigm was refreshing to say the least.
Since you were collecting comments/suggestions: I urge you to take a look at the zipper data structure if you haven't already, they are applicable to what you are working on and are a good intellectual exercise on their own :) In addition, I haven't had the time to go over all of your code but I don't see you mentioned circular dependencies for your tool, is that a feature that is yet to be implemented? In fact this project can probably benefit from drawing ideas from garbage collection techniques since conceptually I think they are very similar.
Cool project. I'm a longtime Haskell skeptic but this article does a great job explaining Haskell's advantages in this useful and otherwise challenging example.
> For example in my case I ran into problem when reading all files lazily. This caused my program to have too many open file handles. It was easily solved though, by hacking a bit to force the complete file to be read directly
This kind of thing is exactly what I'm afraid of and makes me not wanting to commit to a purely functional style. IMO FP is simply the wrong tool for handling expensive resources - e.g. I/O or large memory regions. This is why I think an imperative shell handling these resources, around a functional core (potentially around another tiny imperative innermost core for handling caches), is overall a cleaner approach for performant code.
Laziness (lazy IO, in this case) has nothing to do with a language being functional or not. It has everything to do with a language being strict or lazy. There are strict functional languages. Most are, in fact.
Lazy/strict is just one property that can be problematic in a high performance context. The biggest one to me is mutability. Basically it's a simple test: Can I swap pointers? (e.g. 'model_current' and 'model_next_timestep'). If not, I can't use it, it would slow down numerical solvers tremendously to allocate and free the required memory for each step. However, if I can just have the time iteration in an imperative shell that allows pointer swapping, while keeping the interesting mathematics in a purely functional core, that would be an interesting architecture (because it could make use of inherent parallelism in a better way). So far I haven't seen anything like that becoming truly competitive with Fortran/C/C++ in the HPC space, which I find a shame.
Programming for HPC should be like programming in the future, as Alan Kay likes to say, but it seems to me there is a stark (and IMO unnecessary) disconnect between the worlds of HPC and desktop/web programming today.
There are various ways get the effect (with similar space and performance chacteristic), but there are different approaches for different FP languages. For example, persistent data structures, non-pure primitives (eg Clojure's atoms), monads, etc. For Haskell, these sound relevant if you're set on using strict dense matrices: https://wiki.haskell.org/Monad/SThttps://hackage.haskell.org/package/bed-and-breakfast
I suspect HPC will always be more gnarly than elegant, since by definition HPC is about spending money on making code go fast in very specialized apps.
could you expand on persistent data structures? How do they give me 'free' memory with neither allocation nor mutable data? Or is that outside of pure FP?
So persistent data structures are conceptually just an extension of the classic lisp cells: You can have a pointer to (a, b, c) and a pointer to (a, b, c, d) simultaneously, without using twice the memory or time. This can be extended to trees. And you can for example make an efficient tree-backed vector, with chunks of values at the nodes. With some thought you can even get good atomic properties so you can have safe parallel operations on the same persistent vector.
I'm not very literate with Haskell, but didn't OP have to do a quirky hack with reading the length of the file into the void in order to get rid of open file handles? That doesn't seem to me like Haskell is well designed at letting me take over resource management if the need arises.
It's easily avoided: Just never use the lazy I/O functions. (Not sure if hlint perhaps has a warning you can apply here. It probably should.)
There are real non-brittle solutions to streaming I/O, pipes[1], conduit[2], or io-streams[3] being the most popular AFAICT. (The latter is probably the easiest to use, but is somewhat less 'composable' than the other two, especially if you have monad transformer stacks, etc.) Either that, or just use strict I/O if you're just working with smallish files.
Essentially lazy I/O was a mistake that's unfortunately very hard to fix nowadays without presumably breaking a lot of working-but-potentially-brittle code. Perhaps the time is ripe for starting to actually deprecate them officially... at least almost everyone in the Haskell community seems to agree that it was a mistake and they are a constant trap for newbies.
I wouldn't want to see lazy IO disappear altogether because if you are just processing a file line by line then its a quick and dirty solution that will work fine in practice. However I agree it should come with health warnings and pointers to more scaleable solutions.
Note that it’s possible to give type signatures for definitions in “where” clauses:
However, you may need the ScopedTypeVariables extension[1] in order to be able to express the correct type signature. (This extension will probably be standardised at some point.)The biggest real problem I’ve found with large “where” clauses is that they tend to have a lot of implicit dependencies on variables from the enclosing scope. Sometimes you want that for convenience, readability, or performance, but lifting local definitions out into top-level definitions can also help make them more explicit and reusable.
[1]: https://ocharles.org.uk/blog/guest-posts/2014-12-20-scoped-t...