I deleted my comment because I realize the benchmarks don't cover a 5 minute test. It's unproven whether the Java implementations need to in fact GC over that time. I'm skeptical though -- they probably reuse all their objects, and I think it would be interesting to prove that.
Also, the benchmarks do in fact test all the way up to 16,384 simultaneous connections.[1]
I bet if you published a test case that proves Java GC impacting the max latency over 5 minutes, that would get the attention of TechEmpower and make a case for extending the test past 15 seconds.
> It's unproven whether the Java implementations need to in fact GC over that time. I'm skeptical though -- they probably reuse all their objects
Are you saying that it's common for java web-frameworks to automatically reuse objects and that they do not cause any GC?
> Also, the benchmarks do in fact test all the way up to 16,384 simultaneous connections.
Read jlouis comment once more. It's not that they don't try many connections at once, it's that they take the best result that they get. Big difference.
EDIT:
Looking at the code for the Java Netty/JSON test shows that it does not reuse objects. The biggest takeaway from these tests is that it's hard to do performance testing...
Also, the benchmarks do in fact test all the way up to 16,384 simultaneous connections.[1]
I bet if you published a test case that proves Java GC impacting the max latency over 5 minutes, that would get the attention of TechEmpower and make a case for extending the test past 15 seconds.
[1] "The high-concurrency plaintext test type is tested at 256, 1,024, 4,096, and 16,384 client-side concurrency." http://frameworkbenchmarks.readthedocs.org/en/latest/Project...