If I read the first mistake correctly, then getFeatureFlag() has the side-effect to count how often it was called and uses this to calculate the outcome of the experiment? Wow. I don't know what to say....
Yeah I felt that way too. Initially I thought I wasn't sure what I was missing, since the only difference is that the order of the checks is switched, and the function will still return the same true/false in both cases. Then I thought about side effects and it felt icky.
That's how every one of these tools works, that’s the whole point of using them: you only call them when you’re going to actually show the variation to the user. If you’re running a test that modifies the homepage only, you shouldn’t be calling that decision method in, say, your global navigation code that you show everyone. Or, for instance, if your test only affects how the header looks for subscribers, you have to put an outer if statement “if subscriber“ before the “if test variation.“ How else would it correctly know exactly who saw the test?
When you call the feature flag, it’s going to put the user into one of the groups. The article is saying you don’t want to add irrelevant users (in the example, ones that had already done the action they were testing) because it’s going to skew your results.
