Hey all. Cluster is proxmox 3.1 all up-to-date. I am running into an issue where the cpu performance is degrading. The only way I can get things back in order is to reboot the VM. Everything will be running great and then we take a significant hit on our software encryption performance and notice the process is running very hard 80-90% cpu within the VM. Once this happens I can see the performance difference when running a 7zip benchmark test. There is only one VM on the cluster which is a fresh standard CentOS6.4 load. I am using cpu type "Host" as I need to pass aes-ni to the guest.
When this takes place I am caching our database files into ram with a small utility called "vmtouch". I can reproduce it by also just simply copying the database to /dev/shm/.
Before Performance Degrades
After Performance Degrades
When this takes place I am caching our database files into ram with a small utility called "vmtouch". I can reproduce it by also just simply copying the database to /dev/shm/.
Code:
root@testprox1:~# pveversion -v
proxmox-ve-2.6.32: 3.1-114 (running kernel: 2.6.32-26-pve)
pve-manager: 3.1-24 (running version: 3.1-24/060bd5a6)
pve-kernel-2.6.32-26-pve: 2.6.32-114
pve-kernel-2.6.32-23-pve: 2.6.32-109
lvm2: 2.02.98-pve4
clvm: 2.02.98-pve4
corosync-pve: 1.4.5-1
openais-pve: 1.1.4-3
libqb0: 0.11.1-2
redhat-cluster-pve: 3.2.0-2
resource-agents-pve: 3.9.2-4
fence-agents-pve: 4.0.0-2
pve-cluster: 3.0-8
qemu-server: 3.1-8
pve-firmware: 1.0-23
libpve-common-perl: 3.0-9
libpve-access-control: 3.0-8
libpve-storage-perl: 3.0-18
pve-libspice-server1: 0.12.4-2
vncterm: 1.1-6
vzctl: 4.0-1pve4
vzprocps: 2.0.11-2
vzquota: 3.1-2
pve-qemu-kvm: 1.4-17
ksm-control-daemon: 1.1-1
glusterfs-client: 3.4.1-1Before Performance Degrades
Code:
RAM size: 670445 MB, # CPU hardware threads: 16
RAM usage: 6378 MB, # Benchmark threads: 16
Dict Compressing | Decompressing
Speed Usage R/U Rating | Speed Usage R/U Rating
KB/s % MIPS MIPS | KB/s % MIPS MIPS
22: 47823 2311 2013 46522 | 559050 2433 2071 50398
23: 47367 2470 1953 48261 | 581643 2403 2214 53196
24: 44950 2393 2019 48331 | 648811 2752 2186 60177
25: 46133 2721 1936 52673 | 620536 2735 2133 58343
----------------------------------------------------------------
Avr: 2474 1980 48947 2581 2151 55529
Tot: 2527 2066 52238Code:
7-Zip (A) 9.13 beta Copyright (c) 1999-2010 Igor Pavlov 2010-04-15
p7zip Version 9.13 (locale=en_US.UTF-8,Utf16=on,HugeFiles=on,16 CPUs)
RAM size: 670447 MB, # CPU hardware threads: 16
RAM usage: 3402 MB, # Benchmark threads: 16
Dict Compressing | Decompressing
Speed Usage R/U Rating | Speed Usage R/U Rating
KB/s % MIPS MIPS | KB/s % MIPS MIPS
22: 28145 1218 2248 27380 | 362930 1387 2360 32725
23: 30272 1320 2337 30844 | 342470 1323 2367 31324
24: 28649 1345 2290 30804 | 333863 1383 2238 30967
25: 28538 1402 2324 32584 | 329607 1354 2289 30992
----------------------------------------------------------------
Avr: 1321 2300 30403 1362 2313 31502
Tot: 1341 2307 30952