Huge increase in I/O load on NVMe disks (at equal VM load) after upgrade from Ceph 12 to 15

Hi! Last night we upgraded our production 9-nodes cluster from PVE 5.4 to PVE 6.4 and from Ceph 12 to 14 and then to 15 (Octopus), following the official tutorials. Everything went smoothly and all running VMs have been online during the upgrade, so we're very happy about the operation. Now the cluster and Ceph are all HEALTH_OK and stable, no rebalancing or recovery in process.
But our monitoring system (which is Zabbix-based) is telling us that the OSDs (which are all NVMe SSDs, 4 on each...

Read more