> To be able to tell if Scala 3 itself is faster, we would have to rollback to Scala 2 and try it with the proper JVM tuning. I'm not willing to do that, sorry! Once you've tried Scala 3, there's no going back.
Pardon my ignorance. Why can't you set up a performance test in both environments and measure the speed?
Setting up reliable and reproducible performance environments is as much work as running good quality prod systems. It requires monitoring thing rarely available to monitor, extending prod metrics, creating snapshots, running repeatable tests on different scales. This isn't work for a single person. I'm in a team of 7 working to acquire next customer who will be on average 5x bigger than peaks of our current biggest customer. We've been designing, configuring and creating metrics on EKS since September, alongside training devs and developing performance engineering framework in k6. Lichess doesn't need that much effort and accuracy, but it's still lots of work. Did you know Fargate will schedule you Intel Skylake or Broadwell and there's even 30% difference in performance between them on certain tasks?
> Pardon my ignorance. Why can't you set up a performance test in both environments and measure the speed?
Everyone has a performance test environment, not everyone has one that's separate from production. In this case, generating realistic test patterns is probably very difficult, so you get one environment for everything.
I think it's a matter of effort. Many performance bottlenecks only really show up when you scale up to massive levels. While it's possible to replicate that on a test environment, it takes time and effort to set that up. While doing that is worth it for some applications, I can see why the Lichess developers didn't want to bother.
Time, effort and spare resources that Lichess had better allocate somewhere else.
If Thibault enjoys Scala 3 and feels more productive, while the overall performance on the same hardware is approximately equal, it's already a win for the project.
Pardon my ignorance. Why can't you set up a performance test in both environments and measure the speed?