I'd like to bring up an old issue regarding lvm striping. It's been discussed about in this thread.
http://forum.proxmox.com/threads/698...h-multipath%29
I've been experimenting with drbd a bit and got interesting results. I have two servers with two 1TB SATA disks each and dual 1 Gbit/s NICs bonded together with 802.3ad. Both disks are setup as their own drbd device and traffic is going in separate vlans. Volume group is created over these drbd devices. If VM disk is created manually with '--stripes 2' parameter I can see that drbd traffic is nicely balanced over different NICs and transfer speeds are pretty nice. This is obviously not the case without the parameter.
Host is Intel Q6600 with 8GB RAM
VM has 2 vcores and 1 GB of vRAM with minimal debian 7 installed.
DRBD 8.4.3
Test are made with bonnie
Default (no cache) VM disk. Large 16 GB test size
Smaller test without bonnie -s parameter
Same tests with writeback cache
16 GB
without -s. Doesn't make that much sense because of the host cache but still the writes are replicated to the other DRBD host.
Althought thats not superfast enterprise stuff it is still replicated/shared storage with slow SATA disks. So would it be possible to have stripes and stripesize parameters? Not hard coded but simple input box in the LVM group storage creation form.
Thanks,
FinnTux
http://forum.proxmox.com/threads/698...h-multipath%29
I've been experimenting with drbd a bit and got interesting results. I have two servers with two 1TB SATA disks each and dual 1 Gbit/s NICs bonded together with 802.3ad. Both disks are setup as their own drbd device and traffic is going in separate vlans. Volume group is created over these drbd devices. If VM disk is created manually with '--stripes 2' parameter I can see that drbd traffic is nicely balanced over different NICs and transfer speeds are pretty nice. This is obviously not the case without the parameter.
Host is Intel Q6600 with 8GB RAM
VM has 2 vcores and 1 GB of vRAM with minimal debian 7 installed.
DRBD 8.4.3
Test are made with bonnie
Default (no cache) VM disk. Large 16 GB test size
Code:
Version 1.96 ------Sequential Output------ --Sequential Input- --Random-
Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
changeme 16G 544 98 98631 18 74557 14 2964 97 247750 26 237.9 7
Latency 16467us 741ms 725ms 14770us 25376us 972ms
Version 1.96 ------Sequential Create------ --------Random Create--------
changeme -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
16 21926 37 +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++
Latency 441us 534us 905us 503us 46us 971us
1.96,1.96,changeme,1,1374168637,16G,,544,98,98631,18,74557,14,2964,97,247750,26,237.9,7,16,,,,,21926,37,+++++,+++,+++++,+++,+++++,+++,++
+++,+++,+++++,+++,16467us,741ms,725ms,14770us,25376us,972ms,441us,534us,905us,503us,46us,971usCode:
Version 1.96 ------Sequential Output------ --Sequential Input- --Random-
Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
changeme 2G 549 98 166223 55 102893 20 2782 96 247848 24 441.5 10
Latency 28789us 129ms 145ms 18424us 18763us 71716us
Version 1.96 ------Sequential Create------ --------Random Create--------
changeme -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
16 32242 56 +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++
Latency 458us 560us 819us 432us 36us 1168us
1.96,1.96,changeme,1,1374174958,2G,,549,98,166223,55,102893,20,2782,96,247848,24,441.5,10,16,,,,,32242,56,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,28789us,129ms,145ms,18424us,18763us,71716us,458us,560us,819us,432us,36us,1168usSame tests with writeback cache
16 GB
Code:
Version 1.96 ------Sequential Output------ --Sequential Input- --Random-
Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
changeme 16G 520 97 90191 20 62590 15 2980 97 246269 30 349.1 11
Latency 29155us 427ms 477ms 8851us 37501us 3060ms
Version 1.96 ------Sequential Create------ --------Random Create--------
changeme -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
16 25593 44 +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++
Latency 428us 589us 895us 473us 33us 986us
1.96,1.96,changeme,1,1374173786,16G,,520,97,90191,20,62590,15,2980,97,246269,30,349.1,11,16,,,,,25593,44,+++++,+++,+++++,+++,+++++,+++,+
++++,+++,+++++,+++,29155us,427ms,477ms,8851us,37501us,3060ms,428us,589us,895us,473us,33us,986usCode:
Version 1.96 ------Sequential Output------ --Sequential Input- --Random-
Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
changeme 2G 550 98 129077 50 133918 69 3002 97 856944 72 3651 71
Latency 27009us 250ms 335ms 2911us 7753us 14925us
Version 1.96 ------Sequential Create------ --------Random Create--------
changeme -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
16 31272 55 +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++
Latency 16458us 555us 935us 470us 34us 910us
1.96,1.96,changeme,1,1374171575,2G,,550,98,129077,50,133918,69,3002,97,856944,72,3651,71,16,,,,,31272,55,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,27009us,250ms,335ms,2911us,7753us,14925us,16458us,555us,935us,470us,34us,910usAlthought thats not superfast enterprise stuff it is still replicated/shared storage with slow SATA disks. So would it be possible to have stripes and stripesize parameters? Not hard coded but simple input box in the LVM group storage creation form.
Thanks,
FinnTux