Problems with two containers on two hypervisors

I have two containers, on two hosts. They are paired application servers. Approximately one hour ago the application reported failures.

Looking at the containers I find that one cannot execute common unix commands (example vi). The other I cannot console in - 'vzctl enter $VMID' puts me right back to the proxmox host console.

Normally I'd say 'delete the container(s) and rebuild' however there is some key data inside the containers that I really need to recover. Help ...






root@tn-proxmox-1:~# vzctl --verbose stop 102
Stopping container ...
Killing container ...
Container was stopped
Container is unmounted
root@tn-proxmox-1:~# vzctl --verbose start 102
Starting container ...
Container is mounted
Found osrelease 2.6.32 for dist ubuntu-12.04-x86_64.tar.gz
Running container script: /etc/vz/dists/scripts/debian-add_ip.sh
Got signal 4
Setting CPU units: 1000
Setting CPUs: 16
Running container script: /etc/vz/dists/scripts/debian-set_hostname.sh
Got signal 4
Configure veth devices: veth102.0
Adding interface veth102.0 to bridge vmbr0 on CT0 for CT102
Container start in progress...
root@tn-proxmox-1:~#
root@tn-proxmox-1:~# vzctl enter 102
entered into CT 102
exited from CT 102
root@tn-proxmox-1:~#




root@tn-proxmox-2:~# vzctl --verbose stop 103
Stopping container ...
Container was stopped
Container is unmounted
root@tn-proxmox-2:~# vzctl --verbose start 103
Starting container ...
Container is mounted
Found osrelease 2.6.32 for dist ubuntu-12.04-x86_64.tar.gz
Running container script: /etc/vz/dists/scripts/debian-add_ip.sh
Setting CPU units: 1000
Setting CPUs: 16
Running container script: /etc/vz/dists/scripts/debian-set_hostname.sh
Running container script: /etc/vz/dists/scripts/set_dns.sh
Running container script: /etc/vz/dists/scripts/set_ugid_quota.sh
/bin/bash: line 474: 55 Segmentation fault /usr/sbin/update-rc.d vzquota remove > /dev/null 2>&1
Configure veth devices: veth103.0
Adding interface veth103.0 to bridge vmbr0 on CT0 for CT103
Container start in progress...
root@tn-proxmox-2:~# vzctl --verbose enter 103
Entering CT
entered into CT 103
root@tn-es-node-2:/# ls
apps bin boot dev etc fastboot home lib lib64 lost+found media mnt opt proc root run sbin selinux srv sys tmp usr var
root@tn-es-node-2:/# vi /tmp/foo.txt
Segmentation fault
root@tn-es-node-2:/#