Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

What's the BEAM languages' FFI situation like? I've got the impression that the more higher-level concepts a language supports, the worse it is to do FFI.


Complicated. There are essentially 5 possibilities:

- ports are subprocesses acting as erlang processes

- erl_interface is a more efficient version of the same as the communication uses BEAM's external term format

- C nodes are what it sounds like, basically a separate program acting as a node in a beam cluster

- port drivers have a shared library acting as a process, it's way more efficient but jettisons safety as a crash in the library kills the runtime

- finally NIFs are synchronous functions called in the context of an existing process, they are the fastest option but not only do they kill the emulator they can also screw up the VM state, and they can't be managed by the scheduler without their cooperation (which can severely degrade system stability), BEAM has a concept of "dirty NIFs" which are long-running, can't be suspended, and can't be threaded, if those are appropriately flagged they are run on dedicated "dirty schedulers", dirty schedulers have more overhead but less restrictions, in essence cgo is a more impactful version of that (although I believe dirty schedulers were invented a while after cgo existed, previously "dirty nifs" would just see you drawn and quartered)


I'd say (clarify?) that giving an answer to "What's the BEAM languages' FFI situation like?" is more complicated than actually using most of these options. Once you choose one, implementation ranges from "trivial" to "a little obscure".

Ports just wrap executables you can feed STDIN to and get STDOUT from in order to treat those streams as messages. You make one with one line of code, they behave just like any BEAM process. (you can message them, kill them, etc). If your goal is to use an existing DLL, this involves some C glue code, which looks like what you'd do if you wanted to be able to call the DLL functions in a bash script (take in STDIN, translate strings to appropriate types, call function, return strings to STDOUT).

The erl_interface option is using that library to replace the "translate to/from strings" task with "translate to/from BEAM types". Still a fair amount of glue code for "I want to call this DLL function", but I feel like it should be possible to codegen a lot of it. That might exist already, and if it doesn't it sounds like the kind of fun project I might pick up.

C nodes are using erl_interface to its fullest, defining a full blown BEAM node with one or more processes running on it in C. In practice, this means you can send messages to other BEAM processes, rather than going through an intermediary Port process. It's definitely the most involved option, but it's well documented (like everything else in OTP).

Port drivers free you from the concept of messages within the (even smaller) C code: You make a DLL (that links your target DLL) that provides some mapping information and some dispatch code, then in your BEAM language handle all the "send/receive messages" stuff associated with being a BEAM process. The BEAM node crashing if the library crashes is rightfully considered a significant issue, but it's worth noting that we only care about that so much because the BEAM spoils us with much better safety normally. In any other programming language, a crash in your linked library would probably be expected to cause your entire application to crash, whereas in BEAM land we can even mitigate this by putting our risky code on a different BEAM node, running on the same or different machine, to limit our blast radius.

NIFs allow you to present your C DLL function call as a normal, synchronous BEAM function call. They require the least C glue code, but if any of those calls take more than a millisecond or so you start getting into "thar be dragons" territory on the clean scheduler, or require the use of the dirty scheduler which slows everything else down.

Ultimately, the punchline to all this is that if you want to call an existing shared library from the BEAM, you're going to have to write some amount of C, ranging from "a couple lines per function you want to call" to "defining a small runtime that handles dispatch based on strings".




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: