> - this order of compilation influences outcome and can lead to buggy code
Do you have an example of this? It's my understanding that the big thing about both Erlang and Elixir are that they're functional languages, it doesn't matter what's compiled when. Is this some nightmare compile-time code manipulation thing?
> ...searching [the docs] shows random versions and there isn’t even a link to „open newest”
Is scrolling to the top of the list contained in the version selector near the top left of hexdocs.pm not good enough? If not, why not?
They are probably talking about the child spec needing a unique ID if it's the same module being started.
The docs problem is more of a Google problem. For some reason Google still only shows the 1.12 docs for a lot of searches. The sidebar issue was fixed more recently, I think in the last year. But basically the sidebar wouldn't get loaded until Mermaid finished loading, so it was updated to defer loading of Mermaid. The latest version of ExDoc shouldn't have this problem.
I can spawn as many of those guys as I like and they all become children of the named supervisor. The named supervisor is a 'simple_one_for_one' supervisor with a 'temporary' restart policy.
I guess the thing that might trip folks up with how the docs are worded is not noticing this further up in the document
A supervisor can have one of the following restart strategies specified with the strategy key in the above map:
...
* simple_one_for_one - A simplified one_for_one supervisor, where all child processes are dynamically added instances of the same process type, that is, running the same code.
and that 'start_child/2' accepts EITHER a 'child_spec()' OR a list of terms
and that only the 'child_spec()' type can have an identifier, so the first bullet point in the list of three in the function documentation does not apply.
Also, I find the way the docs USED to print out function types a bit easier to understand than the new style: <https://web.archive.org/web/20170509120825/http://erlang.org...>. (You will need to either close the Archive.org nav banner or scroll up a line to see the first line of the function type information, which is pretty informative.)
I'm talking about the behavior of the one_for_one supervisor:
defmodule Testing.Application do
use Application
@impl Application
def start(_type, _args) do
children = []
opts = [strategy: :one_for_one, name: Testing.Supervisor]
Supervisor.start_link(children, opts)
end
end
defmodule Testing.Server do
use GenServer
def start_link(_), do: GenServer.start_link(__MODULE__, [])
@impl GenServer
def init(_), do: {:ok, nil}
end
When you try to start more than one child, it fails:
> I'm talking about the behavior of the one_for_one supervisor:
Oh, sure, you can vary the ID in non-'simple_one_for_one' supervisors there to make that work. Apologies for inducing you to write out all that transcript and code.
But, OP's claim was:
> - a lot of knowledge is implicit (try checking if you can dynamically add children to a Supervisor)
which is just not fucking true no matter how you slice it. It's true that the relevant documentation doesn't literally say "Calling 'start_child/2' is valid for any kind of 'supervisor'. That's why it's here... to dynamically add children to a 'supervisor'.", but if one bothers to read the docs on supervisors and the function in question it's clear that that's the entire point of 'start_child/2'.
You’re coming from position of knowledge. I don’t memorize documentation because it changes often and I use 3-5 various languages over the month. For fun and profit ;)
So I go into docs, Elixir because that’s the primary source and when I search for „dynamic” and „supervisor” everything points to (no fanfares) DynamicSupervisor. And yet I look at the diff where 12 months earlier my colleague changed DynamicSupervisor to Supervisor because connection adapter tended to crash and not start back up and that day I debug zombie connections.
Erlang has it clearly explained, and ultimately, over couple hours of research I found solution and fixed it, but there was no warning that between two lines of shutdown and delete BEAM could restart a child leading to a shadowing (and - in that case - unmanaged as adapter manager lives in other app that has a slot for a single process only) zombie connection handler.
This is the stuff people rarely look at because many systems don’t have high idempotency requirements but (only a parabole, that’s not my industry) would you like your system to administer double dosage of potentially lethal drugs? None is preferable but it’s still far from happy system land.
As for Hexdocs.pm - this goes once again about operating cost. Yes, I can do this, but search is not great and I more often have a broad queries and search for patterns or guidelines. I rely on „map mode” (how I can find knowledge) and not on „collect mode” (i.e. I keep knowledge), due to some mostly intrinsic traits. And thanks to wonders of modern human tracking^W^Wtechnology I can show anecdotal data of the impact.
When working with Elixir I spent (one random, recent day) 40% of my time on browsing Hexdocs and 60% in editor+console.
One random recent day on Go I spent 95% time in E+C and 5% in documentation.
My first line of code written in Go was less than 6 months ago, my first line of Elixir was written years ago, I would’ve to check my resume when exactly, but somewhere like a 8 years and I toyed with it when it wasn’t yet deemed production ready.
> The process, if any, is terminated and, unless it is a temporary child, the child specification is kept by the supervisor. The child process can later be restarted by the supervisor.
In the English-language docs, the warning is very clear: children that a supervisor will restart on termination may be restarted before you get around to calling 'delete_child'. Children with a restart type of 'permanent' will always cause a race between 't_c' and 'd_c'. Children with a restart type of 'transient' will cause a race if they terminate abnormally... which is a term defined by the same document that the warning comes from.
You keep insisting that this stuff isn't documented. Are you perhaps reading a poor translation of the docs?
> You’re coming from position of knowledge.
No, I'm not. I've forgotten most of the details I ever knew about how the system works and only remember broad strokes. My experience with Erlang/OTP was scattered throughout my spare time over an eight to twelve month period ten years ago. Unlike you, I've never been paid to work with it, and you've worked with it far more recently than I.
The reason I was able to direct you to the right part of the docs was because I said to myself "Wow, it would be fuckin stupid if you couldn't dynamically add children to a supervisor. I remember the Erlang docs being really, really good, so let's see if they failed to describe if and/or how this works.", and then spent like five minutes reading the docs and another couple cross-checking with the Elixir docs.
> I don’t memorize documentation because it changes often...
Do the rules for "+", "-", and (if present) "%" change often in most major languages when they are used with built-in types? [0] It's always well worth your time to learn the rules for commonly-used, bedrock parts of a language, major libraries, and runtime systems that you intend to use. Bedrock parts don't substantially change, because if they did, they would invalidate every program ever written against those parts.
OTP provides a collection of bedrock parts, and supervisors are one such part. Memorizing docs is really stupid, but having a solid understanding of how the major things you use work is always worth the time.
Seriously, how would you function as a programmer if you didn't know how addition or string concatenation worked? If you're using OTP's supervisors, having a good understanding of how they function is just as fundamental.
> One random recent day on Go I spent 95% time in E+C and 5% in documentation.
Sure, that makes some sense. OTP is far more complex and robust than anything Go offers, so it's quite a bit quicker to come up to speed with what's documented in the Go official docs than what's in the OTP docs. Also, as someone who has written Go professionally for the last five, ten years, I warn you that you're going to get turbofucked by the things that -if they are documented at all- are documented only in blog posts or random tutorials. Go is an absolute grab bag of poorly-documented sharp edges and surprising behavior.
> I rely on „map mode” (how I can find knowledge) and not on „collect mode” (i.e. I keep knowledge)... [and] I can show anecdotal data of the impact.
The impact of you failing to familiarize yourself with the necessarily-complex tools you chose [1] to use seems pretty clear to me. You got tripped up by documented behaviour that you didn't bother to understand, which caused you to spend hours looking for solutions that would have been clear after a ten minute trip to the documentation for the 'supervisor' module. Given what you've said elsewhere in this subthread about how your project is an "an enormous Jenga tower that's risky to breath around", I suspect that a significant number of your coworkers also refused to familiarize themselves with the tools they use.
> Do the rules for "+", "-", and (if present) "%" change often in most major languages when they are used with built-in types?
In terms of dynamic or in terms of behavior between them? For former - yes they do change, not often, but they do. Even Elixir is right now raising warnings that `-0.0` and `+0.0` will not be equal, which implies also changes in addition and subtraction (e.g. cancelling out event's value in event based system value might impact on system's behavior).
If that's the latter then it deserves blog post on its own, because some can add mixed types, some are casting in specfic way, some are copying data, some are mutating data, some are doing heuristic casting, some are crashing, some leak memory, some allow modifying pragmas, some allow implicit overloading.
It's a jungle out there. ...and it reminds me about academic joke that '2 + 2 = 5 given extremely high values of 2' - funny one until you spend night trying to figure out why this happens and another two planning vengeance on person who decided that int->float->int is a good trick to use helpful float-taking function on an otherwise perfectly fine integer.
The worth of remembering is a concept I perceive a consideration a cost, usefulness and available memory space. I rather remember that in Elixir/BEAM child shutdown and removal is message driven (and thus can cause race condition) than whether I need to use `+` or `++` for concenating lists.
It's a good thing I asked specifically about built-in types in a particular system, and didn't ask about comparisons between operators in different languages.
> For former - yes they do change, not often, but they do. Even Elixir is right now raising warnings that `-0.0` and `+0.0` will not be equal...
Sure, that's a change to optional behavior to comparison of floating-point zeros. That doesn't change how equality testing, addition, subtraction, or -if available- modular arithmetic works.
As I said:
> It's always well worth your time to learn the rules for commonly-used, bedrock parts of a language, major libraries, and runtime systems that you intend to use. Bedrock parts don't substantially change, because if they did, they would invalidate every program ever written against those parts.
> > It's always well worth your time to learn the rules for commonly-used, bedrock parts of a language, major libraries, and runtime systems that you intend to use.
I disagree. I've been long enough around to see languages sunsetted, libraries sunsetted. Big systems are standing on shamefully old versions. If your job is to work on one language - I agree, but when working with 100s of systems that go over multiple OTP versions, multiple Elixir versions, sprinkled with JavaScript, TypeScript, Ruby 1.0, Elm, Java, "oh my dear is it Python 2 running CoffeeScript?!", then memorizing anything is pointless, because chance is that thing that you memorized is:
- not yet in this project
- no longer in this project
- that tech isn't in the project
- project is written in Malboge, everything you know is irrelevant
- is explicitly forbidden by code owner (for more or less sensible reason)
> Bedrock parts don't substantially change, because if they did, they would invalidate every program ever written against those parts.
Been there, done that, bought a t-shirt. I dislike TypeScript for exact that reason [0], but in Elixir the same is true if you rely on --warnings-as-errors flag due to (in my opinion) broken deprecation mechanism.
Software is full of leaky abstractions. Do you know that it's not guaranteed that your system clock is monotonic? [1]
> Do you know that it's not guaranteed that your system clock is monotonic?
Yes. Wall-clock time is adjustable. That's why there's a monotonic clock function on any serious OS that's running on hardware that makes such a function possible.
> ...but when working with 100s of systems that go over multiple OTP versions [it's not worth understanding how anything that's bedrock works]...
Hey, look at this description and warning in 'terminate_child/2'
> Tells the supervisor SupRef to terminate the child process corresponding to the child specification identified by Id. The process, if there is one, is terminated but the child specification is kept by the supervisor. This means that the child process may be later be restarted by the supervisor. The child process can also be restarted explicitly by calling restart_child/2. Use delete_child/2 to remove the child specification.
Hell, read the rest of that document... notice that the behavior described from nearly twenty years in the past is the same as now. (And I bet you One American Nickel that the behavior described in 1997 is also the same.)
If you don't consider that to be bedrock functionality and worth familiarizing yourself with, I don't know what to tell you.
> The impact of you failing to familiarize yourself with the necessarily-complex tools you chose [1] to use seems pretty clear to me. You got tripped up by documented behaviour that you didn't bother to understand, which caused you to spend hours looking for solutions that would have been clear after a ten minute trip to the documentation for the 'supervisor' module.
As I showed in the other post, this is incorrect (at least in Elixir documentation).
Fortunatelly I read that last, as I'd refrained from further conversation but regarding my situation and memory - I didn't choose so, I was born with a specific type of memory and specific traits. It's useful and it built me a rewarding career. Often problems in software are caused by assumptions and I can't have any. Thanks to that I can work on interesting systems that have hair pulling problems.
However this attitude of both shaming "you should just memorize" and "works for me" approach is one I seen often in Elixir's community and why I don't want to have such conversations in official places. I don't feel a need to be present in environment where I'm not welcome. And yet, peculiarily, I'm often brought as a decision maker regarding recommending choosing or sunsetting technologies and given lack of parameters I do fall back and it wasn't that great.
> However this attitude of both shaming "you should just memorize" and "works for me" approach...
Okay? I'm doing neither, so I don't see why you're bringing that up. I've consistently rebutted your claims that something wasn't explicitly documented by pointing out where it's explicitly documented. I've also called memorization of documentation a fucking stupid thing to do.
> As I showed in the other post, this is incorrect (at least in Elixir documentation).
As I've mentioned in the other post, I don't see how this is incorrect, and await your detailed walkthrough.
> Children with a restart type of 'permanent' will always cause a race between 't_c' and 'd_c'. Children with a restart type of 'transient' will cause a race if they terminate abnormally... which is a term defined by the same document that the warning comes from.
This is not written anywhere explicitly in the docs - I also agree that Erlangs documentation is much better but I’m not saying that Erlang is missing information. I’m talking about Elixir not providing this and marking clearly - because if I need to start reading in Erlang first then why would I layer Elixir on top of it? This is exactly the thing I’m pointing out.
Because your response is long Ill only focus on this point and (hopefully) get back later.
My expectation (implicit) would be that when function is doing 2 lines the messages would be locally ordered. Yes, maybe that’s silly, but in many other languages that’s exactly the case. If I send messages to queue I’m aware that queue might not get two of those. I need to send a transaction, fine. If I broadcast or make a signal/event same happens. But here I have synchronous function with no indication or warnings that it’s a message.
If this can’t be known in documentation, isn’t caught by compiler/analysis, but requires experience or (often) reading source code it is implicit knowledge.
Yes, I posses it too now, but I think it’s a problem.
> This is not written anywhere explicitly in the docs.
It absolutely is. I'll use the Elixir docs as my source:
> A non-temporary child process may later be restarted by the supervisor.
And, further up in the docs when talking about the circumstances under which a supervisor will restart a child that has terminated: [0]
Restart values (:restart)
The :restart option controls what the supervisor should consider to be a
successful termination or not. If the termination is successful, the
supervisor won't restart the child. If the child process crashed, the
supervisor will start a new one.
The following restart values are supported in the :restart option:
:permanent - the child process is always restarted.
:temporary - the child process is never restarted, regardless of the
supervision strategy: any termination (even abnormal) is considered
successful.
:transient - the child process is restarted only if it terminates
abnormally, i.e., with an exit reason other than :normal, :shutdown, or
{:shutdown, term}.
For a more complete understanding of the exit reasons and their impact, see
the "Exit reasons and restarts" section.
And the "Exit reasons and restarts" section says: [1]
> A supervisor restarts a child process depending on its :restart configuration. For example, when :restart is set to :transient, the supervisor does not restart the child in case it exits with reason :normal, :shutdown or {:shutdown, term}.
You go on to say:
> But here I have synchronous function [to affect the state of a supervisor] with no indication or warnings that it’s a message.
Before I get into that, I have two questions for you:
1) How do you affect an Erlang or Elixir process without sending it a message? The docs for Processes [2] don't indicate any other way.
2) Have you never seen or written a function that does not return until it receives the response to an async operation?
Continuing on... from the top of the Supervisor docs, we see:
> A supervisor is a process which supervises other processes, which we refer to as child processes.
"A supervisor is a process...", straight off the bat. That's super clear and explicit, but I'll keep walking through the docs to show you how else this information is communicated to the reader.
If we read on, we see that the first argument to the 'stop_child/2' and 'delete_child/2' functions is of type 'supervisor()', which is defined as '@type supervisor() :: pid() | name() | {atom(), node()}'. What are these? Well, check the docs for how you start a Supervisor. [3] They say three interesting things:
1) The second argument to 'start_link/2' is of type 'option()', which is defined as '{:name, name()}', and 'name()' is defined as 'atom() | {:global, term()} | {:via, module(), term()}' . Keep those types in mind.
2) "If the supervisor and all child processes are successfully spawned (if the start function of each child process returns {:ok, child}, {:ok, child, info}, or :ignore), this function returns {:ok, pid}, where pid is the PID of the supervisor. If the supervisor is given a name and a process with the specified name already exists, the function returns {:error, {:already_started, pid}}, where pid is the PID of that process."
Notice how often it talks about "spawning" the supervisor and returning a PID, and saying that that PID is the PID of the supervisor you just created, or of a named supervisor that already exists.
3) "The options can also be used to register a supervisor name. The supported values are described under the "Name registration" section in the GenServer module docs."
Let's look at the "Name registration" section. [4] I'm not going to quote the whole thing because it'd be a nightmare to reformat sensibly, but the two key sections are
> Both start_link/3 and start/3 support the GenServer to register a name on start via the :name option. Registered names are also automatically cleaned up on termination. The supported values are: an atom ... {:global, term} ... {:via, module, term}...
and the last four items in the bulleted list in the section beginning with
> Once the server is started, the remaining functions in this module (call/3, cast/2, and friends) will also accept an atom, or any {:global, ...} or {:via, ...} tuples. In general, the following formats are supported:
Notice how those bullets match up to the 'name()' type that is passed in to supervisor:start_link/2, and connect that information with the fact that the docs for that function direct you here to learn about how you can register a name for your supervisor. Combine that information with the fact that the first argument to the "Tell the supervisor to do something" functions is of type 'supervisor()' and the fact that 'start_link' returns a PID, and it's really, really clear that a supervisor is another process that you can (optionally) name and refer to by name, rather than PID.
Once we understand that a supervisor is a process, and that the functions to instruct a supervisor to do things require the information required to contact a process, what other conclusion can we draw than "Communications with a supervisor is async, because communications with all processes are async."?
def start_new(name, config) do
# Logging set up
Supervisor.start_child(
name,
{ HandlerModule, config }
)
end
def replace_supervisor(name, config) do
Supervisor.terminate_child(name, HandlerModule) # Success
Supervisor.delete_child(name, HandlerModule) # Failure
start_new(name, config)
end
That is exact code. Success and failure were logged. Also (from Erlang's documentation)
> one_for_one - If one child process terminates and is to be restarted, only that child process is affected. This is the default restart strategy.
In terminate child you can read that (once again Erlang).
> If the supervisor is not simple_one_for_one, Id must be the child specification identifier. The process, if any, is terminated and, [[unless it is a temporary child, the child specification is kept by the supervisor]]. The child process can later be restarted by the supervisor.
Sorry, what happened after or during the call to delete_child/2 that caused you to consider it to have failed?
> So yeah, Elixir documentation is wrong.
I don't see what's wrong about the Elixir documentation. Walk me through it, please? Do remember that the default restart strategy for a supervisor is 'permanent', and that 'one_for_one' only ensures that the supervisor-initiated restart of one supervised child doesn't cause the supervisor to restart any other supervised children.
After tracing the code this is exactly what happened (in this code exactly):
1. Terminate child X
2. /Supervisor restarts X/
3. Delete child X {:error, :running}
4. Supervisor.start_child Y {:ok, PID}
5. /X and Y are both running/
As for incorrectness:
> the supervisor does not restart the child in case it exits with reason :normal, :shutdown or {:shutdown, term}.
`terminate_child` is sending shutdown and yet it's being restarted.
And to emphasise on use case. The child is connection handler. Service node changed. It NEEDS to be restarted on crash, but has to be replaced during handoff.
I believe you start to get into "huh?" mode with me. I have a treasure trove of those. (Btw., in Erlang repository there's plenty of notes mentioning THIS exact behavior and if I didn't overskim - even some bugs caused by it - you can search for terminate_child.
> It NEEDS to be restarted on crash, but has to be replaced during handoff.
I question why you're handing off things between supervisors. If this is something you actually need to do, then 'delete_child/2' so the supervisor doesn't restart the child, terminate the child yourself, and re-start the child on the new supervisor.
EDIT: Actually, no, you can't 'delete_child/2'. You need to change the supervisor type from 'permanent', to the type that does exactly what you say you need. I'll leave it to you to read the docs. /EDIT
> `terminate_child` is sending shutdown and yet it's being restarted.
Here's the context for that partial quote that you pulled from [0]:
> A supervisor restarts a child process depending on its :restart configuration. For example, when :restart is set to :transient, the supervisor does not restart the child in case it exits with reason :normal, :shutdown or {:shutdown, term}.
Re-read that first sentence that you chose to not quote. Then read about the ':restart' supervisor configuration and how it describes when a supervised child is and is not restarted. [1]
> I believe you start to get into "huh?" mode with me.
Yep. Selective quoting when it's trivial for your conversation partner to find the lies by omission definitely put me into "huh?" mode with you.
What? This is very clear: <https://www.erlang.org/doc/apps/stdlib/supervisor.html#start...>, as is this: <https://hexdocs.pm/elixir/1.18.3/Supervisor.html#start_child...>. What am I missing?
> - this order of compilation influences outcome and can lead to buggy code
Do you have an example of this? It's my understanding that the big thing about both Erlang and Elixir are that they're functional languages, it doesn't matter what's compiled when. Is this some nightmare compile-time code manipulation thing?
> ...searching [the docs] shows random versions and there isn’t even a link to „open newest”
Is scrolling to the top of the list contained in the version selector near the top left of hexdocs.pm not good enough? If not, why not?