I assume you are the author? I think you make valid case/points but I have some kind counter points if you don't mind
> 1. Having too many unit tests makes your codebase harder to evolve. You ideally want to have as few tests as possible capture as many properties of your domain as possible.
Yes but I have found what often happened for me with with REPL environments is the actual code base would be littered with stuff to massage the REPL (commented out or left behind). At least with the unit tests that playing around stuff is away from the actual code.
For both cases there is always the delete button :) . Also for some reason many developers I have worked with don't seem to have a problem deleting or putting an ignore on a test. After all the tests are source controlled. I do get your point but I don't think its that strong.
> 2. Tests can only ever answer close-ended questions: "does this work?", but not "how does this work?", "what does this look like?" etc.
I fail to understand this point. I mean you can obviously write tests that just run stuff and not throw an exception or error. Furthermore you can share how you set stuff up with other developers.... and again you can just delete it if its obnoxious.
> 3. Tests typically won't run in real-world conditions: they'll use simple, artificial data and mocks of services such as databases or API clients. As a result, they don't typically help you understand a problem that only happens on real-life data, nor do they give you confidence that the real-life implementations of the services they emulate do work.
This is exactly what I do not like about REPLs. You setup a custom environment and its hard to keep track of what you have done. I don't like the "not repeatable" nature of it. I do think you make excellent points about how immutability helps that problem as stuff basically becomes a log but for other languages this is not the case.
However this is by far your strongest point. There are languages that allow you to play with a system while its running. Perhaps not through the command line but through a debugger. The Java debugger in Eclipse/IntelliJ can evaluate expressions and are not far off from being REPLs.... in some cases the debuggers are stronger than REPLs.
>> 2. Tests can only ever answer close-ended questions: "does this work?", but not "how does this work?", "what does this look like?" etc.
> I fail to understand this point. I mean you can obviously write tests that just run stuff and not throw an exception or error. Furthermore you can share how you set stuff up with other developers.... and again you can just delete it if its obnoxious.
Yes, but the point is that a test mostly gives you a Yes/No answer, not a visualization. What's more, sometimes you need to set up a fair amount of state as you explore (think of the examples in the video, where you call an external API based on a previous result of that external API etc.) - not something that is convenient to do in a test.
> > 3. Tests typically won't run in real-world conditions: they'll use simple, artificial data and mocks of services such as databases or API clients. As a result, they don't typically help you understand a problem that only happens on real-life data, nor do they give you confidence that the real-life implementations of the services they emulate do work.
> This is exactly what I do not like about REPLs. You setup a custom environment and its hard to keep track of what you have done. I don't like the "not repeatable" nature of it. I do think you make excellent points about how immutability helps that problem as stuff basically becomes a log but for other languages this is not the case.
It's not that the environment is custom, is that it's real. Do you call your payment service or your mail sending service from your tests? This is exactly the kind of thing you want to experiment with in a supervised, non-repeatable way.
You can put stuff in the REPL, or in your test suite, or both, and there are pitfalls in each case, but at least you have a choice; indeed, you have to use your best judgement to decide what will be persisted, repeated and shared with your team and what will be forgotten after your REPL session end, but I wouldn't call making the wrong choice a deficiency of the REPL, rather an error on the programmer's side.
> It's not that the environment is custom, is that it's real. Do you call your payment service or your mail sending service from your tests? This is exactly the kind of thing you want to experiment with in a supervised, non-repeatable way.
The irony you mentioning that is I have actually written "tests" that call braintree and stripe...
Just to clarify when I say "test" I don't mean some precise definition of "unit test" or "integration test". I mean in a runner that runs your code. Unit tests basically allow you to make a whole bunch of entry points (aka static main(args)).
I have many times setup a particularly environment and then repeatedly ran a unit test against that environment (I then commented or deleted that code later).
I think in large part of what your saying is good about REPLs is that they allow hot code swapping but that is only IMO really one part of the REPL. The key to really good REPL should be human input (think Excel) and human output (think images and graphs). There was a company recently shown on HN called Luna [1] and I think that is what a REPL should be.
And I particularly pick on this because you mention Bret Victor who is (err I guess was) actually working on environments like this.
Otherwise call it a unit test... call it a REPL... call it hot code swapping but in current JVM environments the difference can be pretty nebulous.
> I think in large part of what your saying is good about REPLs is that they allow hot code swapping but that is only IMO really one part of the REPL. The key to really good REPL should be human input (think Excel) and human output (think images and graphs).
I definitely could have spent more time on this part in the article. You may want to have a look at Proto-REPL (https://atom.io/packages/proto-repl), which is one of Clojure's REPLs (for the Atom editor)
> 1. Having too many unit tests makes your codebase harder to evolve. You ideally want to have as few tests as possible capture as many properties of your domain as possible.
Yes but I have found what often happened for me with with REPL environments is the actual code base would be littered with stuff to massage the REPL (commented out or left behind). At least with the unit tests that playing around stuff is away from the actual code.
For both cases there is always the delete button :) . Also for some reason many developers I have worked with don't seem to have a problem deleting or putting an ignore on a test. After all the tests are source controlled. I do get your point but I don't think its that strong.
> 2. Tests can only ever answer close-ended questions: "does this work?", but not "how does this work?", "what does this look like?" etc.
I fail to understand this point. I mean you can obviously write tests that just run stuff and not throw an exception or error. Furthermore you can share how you set stuff up with other developers.... and again you can just delete it if its obnoxious.
> 3. Tests typically won't run in real-world conditions: they'll use simple, artificial data and mocks of services such as databases or API clients. As a result, they don't typically help you understand a problem that only happens on real-life data, nor do they give you confidence that the real-life implementations of the services they emulate do work.
This is exactly what I do not like about REPLs. You setup a custom environment and its hard to keep track of what you have done. I don't like the "not repeatable" nature of it. I do think you make excellent points about how immutability helps that problem as stuff basically becomes a log but for other languages this is not the case.
However this is by far your strongest point. There are languages that allow you to play with a system while its running. Perhaps not through the command line but through a debugger. The Java debugger in Eclipse/IntelliJ can evaluate expressions and are not far off from being REPLs.... in some cases the debuggers are stronger than REPLs.