I have been testing a Proxmox 4.0 node. With previous versions of proxmox, when the power button on the host is pressed, a shutdown command is issued via terminal, or shutdown initiated via gui, the node would send a acpi message to any running guests, and wait the specified or default timeout for each guest to shut down before shutting down the node.
With V4 the when a shutdown is issued, all vms stop almost immediately, and the node shuts down.
What I think I have determined:
* 'pvesh --nooutput create /nodes/localhost/stopall' will cleanly shut down nodes, so will 'service pve-manager stop'. "Stop All VMs" in the gui works OK.
* pve-manager is not installed in /etc/rcX.d/, something is still causing VMs to start up automatically ( possibly via '/etc/init.d/pve-manager start', possibly another way, I see two instances of kvm ID 100 running when the node starts)
Taken from 'ps -elf' during startup:
Code:
4 S root 1094 1 0 80 0 - 1082 wait 00:29 ? 00:00:00 /bin/sh /etc/init.d/pve-manager start
4 S root 1096 1094 27 80 0 - 81021 poll_s 00:29 ? 00:00:00 /usr/bin/perl /usr/bin/pvesh --nooutput create /nodes/localhost/s
1 S root 1123 1096 0 80 0 - 81251 hrtime 00:29 ? 00:00:00 task UPID:pve2:00000463:00000921:55ED9F53:startall::rootpam:
5 S root 1127 1123 0 80 0 - 84640 poll_s 00:29 ? 00:00:00 task UPID:pve2:00000467:00000925:55ED9F54:qmstart:100:root pam:
6 S root 1140 1127 3 80 0 - 65138 pipe_w 00:29 ? 00:00:00 /usr/bin/kvm -id 100 -chardev socket,id=qmp,path=/var/run/qemu-server/100.qmp,server,nowait -mon chardev=qmp,mode=control -vnc unix:/var/run/qemu-server/100.vnc,x509,password -pidfile /var/run/qemu-server/100.pid -daemonize -smbios type=1,uuid=e253ddd4-08c8-4833-9af7-a3c64cc28f93 -name test -smp 8,sockets=1,cores=8,maxcpus=8 -nodefaults -boot menu=on,strict=on,reboot-timeout=1000 -vga cirrus -cpu kvm64,+lahf_lm,+x2apic,+sep,+kvm_pv_unhalt,+kvm_pv_eoi,enforce -m 8192 -k en-us -device pci-bridge,id=pci.2,chassis_nr=2,bus=pci.0,addr=0x1f -device pci-bridge,id=pci.1,chassis_nr=1,bus=pci.0,addr=0x1e -device piix3-usb-uhci,id=uhci,bus=pci.0,addr=0x1.0x2 -device usb-tablet,id=tablet,bus=uhci.0,port=1 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3 -iscsi initiator-name=iqn.1993-08.org.debian:01:512f79e8bf1 -drive file=/var/lib/vz/template/iso/ubuntu-14.04.1-desktop-amd64.iso,if=none,id=drive-ide2,media=cdrom,aio=threads -device ide-cd,bus=ide.1,unit=0,drive=drive-ide2,id=ide2,bootindex=200 -drive file=/var/lib/vz/images/100/vm-100-disk-1.qcow2,if=none,id=drive-virtio0,format=qcow2,cache=none,aio=native,detect-zeroes=on -device virtio-blk-pci,drive=drive-virtio0,id=virtio0,bus=pci.0,addr=0xa,bootindex=100 -netdev type=tap,id=net0,ifname=tap100i0,script=/var/lib/qemu-server/pve-bridge,downscript=/var/lib/qemu-server/pve-bridgedown,vhost=on -device virtio-net-pci,mac=22:08:69:1C:02:B1,netdev=net0,bus=pci.0,addr=0x12,id=net0,bootindex=300
7 R root 1143 1 7 80 0 - 2201354 - 00:29 ? 00:00:00 /usr/bin/kvm -id 100 -chardev socket,id=qmp,path=/var/run/qemu-server/100.qmp,server,nowait -mon chardev=qmp,mode=control -vnc unix:/var/run/qemu-server/100.vnc,x509,password -pidfile /var/run/qemu-server/100.pid -daemonize -smbios type=1,uuid=e253ddd4-08c8-4833-9af7-a3c64cc28f93 -name test -smp 8,sockets=1,cores=8,maxcpus=8 -nodefaults -boot menu=on,strict=on,reboot-timeout=1000 -vga cirrus -cpu kvm64,+lahf_lm,+x2apic,+sep,+kvm_pv_unhalt,+kvm_pv_eoi,enforce -m 8192 -k en-us -device pci-bridge,id=pci.2,chassis_nr=2,bus=pci.0,addr=0x1f -device pci-bridge,id=pci.1,chassis_nr=1,bus=pci.0,addr=0x1e -device piix3-usb-uhci,id=uhci,bus=pci.0,addr=0x1.0x2 -device usb-tablet,id=tablet,bus=uhci.0,port=1 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3 -iscsi initiator-name=iqn.1993-08.org.debian:01:512f79e8bf1 -drive file=/var/lib/vz/template/iso/ubuntu-14.04.1-desktop-amd64.iso,if=none,id=drive-ide2,media=cdrom,aio=threads -device ide-cd,bus=ide.1,unit=0,drive=drive-ide2,id=ide2,bootindex=200 -drive file=/var/lib/vz/images/100/vm-100-disk-1.qcow2,if=none,id=drive-virtio0,format=qcow2,cache=none,aio=native,detect-zeroes=on -device virtio-blk-pci,drive=drive-virtio0,id=virtio0,bus=pci.0,addr=0xa,bootindex=100 -netdev type=tap,id=net0,ifname=tap100i0,script=/var/lib/qemu-server/pve-bridge,downscript=/var/lib/qemu-server/pve-bridgedown,vhost=on -device virtio-net-pci,mac=22:08:69:1C:02:B1,netdev=net0,bus=pci.0,addr=0x12,id=net0,bootindex=300
* Adding a rcX.d script to run either 'pvesh --nooutput create /nodes/localhost/stopall' or 'service pve-manager stop' doesn't help. I've confirmed the script runs, the listed commands end immediately. Either the VMs have already been terminated by then, something else runs simultaneously to kill them, or those commands behave differently during shutdown.
Sample file registered with 'update-rc.d pve-manager-test defaults':
Code:
#!/bin/sh
### BEGIN INIT INFO
# Provides: pve-manager-test
# Required-Start: $remote_fs pve-firewall
# Required-Stop: $remote_fs pve-firewall
# Default-Start:
# Default-Stop: 0 1 6
# Short-Description: PVE VM Manager
### END INIT INFO
. /lib/lsb/init-functions
PATH=/sbin:/bin:/usr/bin:/usr/sbin
DESC="PVE Status Daemon"
PVESH=/usr/bin/pvesh
test -f $PVESH || exit 0
# Include defaults if available
if [ -f /etc/default/pve-manager ] ; then
. /etc/default/pve-manager
fi
case "$1" in
start)
if [ "$START" = "no" ]; then
exit 0
fi
echo "Starting VMs and Containers"
pvesh --nooutput create /nodes/localhost/startall
;;
stop)
echo "Stopping running Backup"
date >> /testfile
echo "Stopping running Backup" >> /testfile
vzdump -stop
echo "Stopping VMs and Containers"
echo "Stopping VMs and Containers" >> /testfile
pvesh --nooutput create /nodes/localhost/stopall
service pve-manager stop
echo "Done." >> /testfile
;;
reload|restart|force-reload)
# do nothing here
;;
*)
N=/etc/init.d/$NAME
echo "Usage: $N {start|stop|reload|restart|force-reload}" >&2
exit 1
;;
esac
exit 0
Am I missing something obvious? Can anyone confirm shutdown behaviour has changed? Does anyone know how to configure nodes to cleanly shut down guests on node shutdown?