Some people have claimed that pure A/B testing is an agent for enshittification, both on a quality and ethical dimension. And I can’t see how those people are particularly wrong.
There are systems out there that can do AB/CD testing and those do a better job of finding pairs of changed that have compounding effects.
You cannot A/B test your way from chocolate and peanut butter to cherry and vanilla. So we get to deal with tone deaf companies who feel their analytics are proving that customers either don’t know what they want or are lying about what they want. But that’s not something A/B testing can prove. It takes more sophisticated experiments than that.
There are systems out there that can do AB/CD testing and those do a better job of finding pairs of changed that have compounding effects.
You cannot A/B test your way from chocolate and peanut butter to cherry and vanilla. So we get to deal with tone deaf companies who feel their analytics are proving that customers either don’t know what they want or are lying about what they want. But that’s not something A/B testing can prove. It takes more sophisticated experiments than that.