I wonder, is increasing temperature of the M.2 NVMe disks affecting the measured performance? Or is P620 cooling system efficient enough to keep temp of the number of disks low?
Both quad SSDs adapters had a fan on it and the built in M.2 ones had a heatsink, right in front of one large chassis fan & air intake. I didn't measure the SSD temperatures, but the I/O rate didn't drop over time. I was bottlenecked by CPU when doing small I/O tests, I monitored the current MHz from /proc/cpuinfo to make sure that the CPU speeds didn't drop lower than their nominal 3.9 GHz (and they didn't).
Btw, even the DIMMs have dedicated fans and enclosure (one per 4 DIMMs) on the P620.
Anyway, thanks for the inspirative post!