> There is no value for logging "function entry/exit, with param values" when all collaborations succeed and the system operates as intended.
Well, I agree completely, but those conditions are a tall order. The whole point of debugging (by whatever means you prefer) is for those situations in which things don't succeed or operate as intended. If I have a failure, and suspect a major subsystem, I sure do want to see all calls and param values leading up to a failure.
In addition to this point, you have constructed a strawman in which logging is on all the time. Have you ever looked at syslog? On my desktop Linux system, output there counts as voluminous. It isn't so much space, or so CPU-intensive that I would consider disabling syslog output (even if I could).
The large distributed system I worked on would produce a few GB per day, and the logs were rotated. A complete non-issue. And for the rare times that something did fail, we could turn up logging with precision and get useful information.
>> There is no value for logging "function entry/exit, with param values" when all collaborations succeed and the system operates as intended.
> Well, I agree completely, but those conditions are a tall order.
Every successful service invocation satisfies "all collaborations succeed and the system operates as intended." Another way to state this is every HTTP `1xx`, `2xx`, and `3xx` response code produced by an HTTP service qualifies as such.
> The whole point of debugging (by whatever means you prefer) is for those situations in which things don't succeed or operate as intended.
Providing sufficient context in the presence of errors, or "situations in which things don't succeed or operate as intended", was addressed thusly:
If detailed method invocation history is a requirement,
consider using the Writer Monad and only emitting log
entries when either an error is detected or in an
"unconditionally emit trace logs" environment (such as
local unit/integration tests).
> If I have a failure, and suspect a major subsystem, I sure do want to see all calls and param values leading up to a failure.
See above.
> In addition to this point, you have constructed a strawman in which logging is on all the time.
No, I addressed your original premise in the context of a production web application, where logging is configured during deployment.
See also your own contradiction by previously asserting, "I sure do want to see all calls and param values leading up to a failure."
So which is it?
Did I construct a strawman "in which logging is on all the time"?
Or do you "want to see all calls and param values leading up to a failure", which requires "logging is on all the time"?
> Have you ever looked at syslog?
This is a strawman. Syslog is a component for logging and has nothing to do with the programs which use it.
> The large distributed system I worked on would produce a few GB per day, and the logs were rotated. A complete non-issue.
If this is the same system you described in a different comment also in this thread, I identified a standard industry practice of log entries produced by application nodes in a production environment being unconditionally sent to a log aggregator and not stored in a local file system. The reasons for this are well documented.
I'm not sure what point you are making with your scenario involving HTTP response codes. What if the HTTP server crashes, and doesn't send a response at all?
I don't know from Writer Monads. But if you only emitting log entries on some future failure or request, then that's potentially a lot of logging to keep somewhere. Where? Log aggregator? Local files? Memory? What about log volume? Does this writer monad implement log rotation? It sounds like you are sweeping a lot of the things you object to under this writer monad rug.
Let me be real clear about all calls and param values leading up to a failure.
- In normal operation, turn logging off completely, or turn on some level that produces tolerable log volume, (it seems like your threshold is much lower than mine).
- When a failure occurs: Restart the service with more logging enabled, (hence the all calls an param values), so that you have logging when the failure occurs again.
About local logs vs a log aggregator: The system I worked on was a shared nothing archive. To add storage and metadata capacity, you add nodes. Each node also stored its own log files. I get that this may not be the answer all the time, and that a log aggregator is useful in some scenarios. However, even in that case, your concerns about log volume seem overblown to me.
> I'm not sure what point you are making with your scenario involving HTTP response codes.
My point was to identify how common the "happy path" scenario is and was in direct response to:
>> There is no value for logging "function entry/exit, with param values" when all collaborations succeed and the system operates as intended.
> Well, I agree completely, but those conditions are a tall order.
Regarding your question:
> What if the HTTP server crashes, and doesn't send a response at all?
Again, the HTTP status codes were used to illustrate the frequency of successful invocations. But to your point, if an HTTP server crashes then log entries for in-flight workflows would likely not be emitted. A similar possibility also holds for local file system logging as buffering exists (sometimes on multiple levels).
> I don't know from Writer Monads.
No worries. All it is is a formal definition of a type having specific capabilities called "laws" in the functional programming world.
> But if you only emitting log entries on some future failure or request, then that's potentially a lot of logging to keep somewhere. Where? Log aggregator? Local files? Memory?
What is meant by "future failure" is a failure potentially encountered during the evaluation of a single invocation. In the context of a HTTP server, this would be the handling of each submission to a specific HTTP endpoint and verb. This is often defined within an IO Monad[0], but does not have to be, and is out of scope for this discussion.
The answer to the rest of your questions is the deferred log entry definitions are held in memory for the duration of the single service invocation, with any log emissions produced transmitted to a network-accesible log aggregator via the logging component used.
> Let me be real clear about all calls and param values leading up to a failure.
The scenario you kindly shared is understandable, yet is one which has been unacceptable in teams I have worked with. Losing the "first error" is not an option in those environments.
Well, I agree completely, but those conditions are a tall order. The whole point of debugging (by whatever means you prefer) is for those situations in which things don't succeed or operate as intended. If I have a failure, and suspect a major subsystem, I sure do want to see all calls and param values leading up to a failure.
In addition to this point, you have constructed a strawman in which logging is on all the time. Have you ever looked at syslog? On my desktop Linux system, output there counts as voluminous. It isn't so much space, or so CPU-intensive that I would consider disabling syslog output (even if I could).
The large distributed system I worked on would produce a few GB per day, and the logs were rotated. A complete non-issue. And for the rare times that something did fail, we could turn up logging with precision and get useful information.