>That would be fine if the core thing needing unit testing was the data generation/ transformation logic, but just as often as not it's the output formatting too.
Output formatting touches io. In this case it is no longer a unit test that touches these things. Unit tests by definition test ONLY internal logic and transformations.
It is literally the definition of unit tests.
When you test things like stdout that becomes an integration test and Not a unit test. It requires some external thing or some global black magic monkey patch that changes what print does to do integration testing.
(Btw making print formatting unit testable means segregating the formatting from the print. Produce the string first, test that, then print, because print can never be unit tested by definition)
Typically programmers segregate these levels of testing because unit tests are easier to write. But to write unit tests your code has to be written in a way to cater to it. Often this style of coding actually improves your code it makes it much more modular. The reason is because pure functions that output data can be composed with all kinds of io functions. You can move it all over the place and to different platforms with different forms of IO. Print has no meaning in certain embedded systems so it can't be moved... By segregating the logic out it makes it so I can move the logic without the io baggage.
Chatgpt 100 percent gets the difference that's why it did what it did. I think you and the OP don't fully understand the meaning of unit testing.
Don't take this the wrong way, but just because you don't know this doesn't say anything about your skills as a programmer. But just recognize that this concept is basic and is pretty much something universal among testing.
> Unit tests by definition test ONLY internal logic and transformations
Output formatting is still a type of transformation! The function explicitly takes the numbers and prints them as decimal integers with newlines between each.
A test to confirm that it IS in that format is still a unit test.
BTW I gave ChatGPT the prompt I would give, and I have to say the answer looks pretty good, even if I'm not a Python programmer and it's not the way I'd do it (which would be to change the function to allow passing in an output stream):
class MyFunctionTestCase(unittest.TestCase):
def test_my_function(self):
expected_output = "0\n1\n2\n"
with patch('sys.stdout', new=StringIO()) as fake_out:
my_function(3)
self.assertEqual(fake_out.getvalue(), expected_output)
With a few more prompts I also managed to get it give me this version:
def my_function(x: int) -> str:
output = ""
for i in range(x):
output += str(i) + "\n"
return output
Which I'd argue somewhat changes the code that was originally written, but it's still a pretty decent answer.
There's no doubt there's some impressive stuff going on that it can do such things, the real issue for me is that when I've tried on far more complex functions it's tended to break down (quite badly in some cases).
>Output formatting is still a type of transformation!
I'll quote part of my reply (which you missed):
(Btw making print formatting unit testable means segregating the formatting from the print. Produce the string first, test that, then print, because print can never be unit tested by definition)
Right? Think about it. You want to unit test your formatting, remove the logic from the atomic IO function. Otherwise you can't test it via a unit test because that's the definition of unit testing. I realize that there is formatting that's part of the internal functionality of printf, but really all that means is that funcitonality can never really be unit tested. If you want to test printf, that happens at the integration level... By Defintion.
>BTW I gave ChatGPT the prompt I would give, and I have to say the answer looks pretty good, even if I'm not a Python programmer and it's not the way I'd do it (which would be to change the function to allow passing in an output stream):
It's wrong in this case. Unless you specifically asked it to write unit testable code, what it did here is write a hack that monkey patches the program. It's a huge hack. It didn't write unit testable code, but rather it wrote a integration test that monkey patches stdout, negating any need to make your code "unit testable" no refactoring needed using this method. The entire concept of refactoring code to be unit testable flies out the door in this case as you're just using integration tests to get around everything.
I mean yeah you use the unit test library but is not technically a unit test. It's fine I'm not a stichler for what style of testing is used in practice but what I am saying is that what chatgpt did previously was literally follow my instructions to the letter. It did it exactly 100% correctly. Think about it. I asked chatgpt to make the Code more unit testable. You didn't have chatgpt do anything to the code. You simply changed the test from a unit test to integration test. Huge difference. I mean if your case was the "proper" way then what does it even mean to make code "unit testable" if you're not even touching the code? Like why does the concept of "making code more unit testable" even exist if we're just changing tests to make everything unit testable? Ask yourself this and you'll realize that the only answer is basically what I just told you previously.
I've been writing unit tests for over 15 years (actually longer, but before that they were just throwaway run-once testing stubs). I wouldn't consider what you got ChatGPT to produce to be an adequate rewrite of a function to make it unit testable (and several others in this thread have expressed the same view). Even the "hack" using monkey patching makes for a more actually-useful test.
I'm perfectly aware of what the difference is, thank you. The function you gave to ChatGPT explicitly requests a stream to format integers as decimals, separated by newlines. The version it gave as being 'unit-testable' did not, and hence wasn't a 'factually correct' answer. In some cases that may be perfectly fine, but in others it most definitely isn't.
Output formatting touches io. In this case it is no longer a unit test that touches these things. Unit tests by definition test ONLY internal logic and transformations.
It is literally the definition of unit tests.
When you test things like stdout that becomes an integration test and Not a unit test. It requires some external thing or some global black magic monkey patch that changes what print does to do integration testing.
(Btw making print formatting unit testable means segregating the formatting from the print. Produce the string first, test that, then print, because print can never be unit tested by definition)
Typically programmers segregate these levels of testing because unit tests are easier to write. But to write unit tests your code has to be written in a way to cater to it. Often this style of coding actually improves your code it makes it much more modular. The reason is because pure functions that output data can be composed with all kinds of io functions. You can move it all over the place and to different platforms with different forms of IO. Print has no meaning in certain embedded systems so it can't be moved... By segregating the logic out it makes it so I can move the logic without the io baggage.
Chatgpt 100 percent gets the difference that's why it did what it did. I think you and the OP don't fully understand the meaning of unit testing.
Don't take this the wrong way, but just because you don't know this doesn't say anything about your skills as a programmer. But just recognize that this concept is basic and is pretty much something universal among testing.