Hi all, quick question (maybe ?)
I am in the process of configuring a storage array for use with a new build proxmox deploy for a client.
- 2 physical nodes running proxmox, latest version, local modest size hardware raid storage for base install of proxmox
- 1 disk array with ~24Tb usable space (Raid6, hotspare, 16x2Tb drives)
- disk array has 2 x fibre ports, so each proxmox node will have a single connection via Fibre to the disk array. LUN(s) on the array will be exposed/visible to both proxVe nodes to facilitate 'shared cluster' access.
- I plan to configure raid volume LUN(s) on the disk array and provision the space to proxmox as shared LVM storage, so live migration is possible easily.
The fun question: I haven't deployed this much shared storage on a proxmox cluster before. (12Tb or less is my current deploy biggest for a project). I'm curious if anyone has strong recommendation about known gotchas in this kind of topology, ie,
- single large LUN of approx 24Tb usable disk space, works well, no gotchas?
- make it visible to both proxVE hosts,
- VM LVMs will be carved off in here by proxmox as disk space is allocated for VMs using this storage pool.
or
- is there a maximum size for the LVM Volume (ie, don't go above 12 Tb per LUN? deploy as 2 x 12Tb instead of 1 x 24Tb ?)
- is there any inherent performance limit which makes LVM get stupid or more PITA to deal with above a certain size ?
I know LVM on 64bit platform supports 'really big' disk capacity in theory. I also know from experience (for example) that ext4 on 64bit is 'meant' to support really big disk capacity but for some platforms, for some support libraries for e2fs utils it means you can't format really big volumes (ie, >12 Tb for example) without having to either compile yourself properly from scratch the disk formatting tools, or other workarounds. So I figure I might as well ask if anyone has strong recommendatoin around this kind of config where I have the choice of deploying a single 24Tb storage volume 'for simplicity' or better to do other things (2x12 Tb, 3 x 8Tb?) for example.
Any comments are greatly appreciated.
Thanks!
Tim Chipman
I am in the process of configuring a storage array for use with a new build proxmox deploy for a client.
- 2 physical nodes running proxmox, latest version, local modest size hardware raid storage for base install of proxmox
- 1 disk array with ~24Tb usable space (Raid6, hotspare, 16x2Tb drives)
- disk array has 2 x fibre ports, so each proxmox node will have a single connection via Fibre to the disk array. LUN(s) on the array will be exposed/visible to both proxVe nodes to facilitate 'shared cluster' access.
- I plan to configure raid volume LUN(s) on the disk array and provision the space to proxmox as shared LVM storage, so live migration is possible easily.
The fun question: I haven't deployed this much shared storage on a proxmox cluster before. (12Tb or less is my current deploy biggest for a project). I'm curious if anyone has strong recommendation about known gotchas in this kind of topology, ie,
- single large LUN of approx 24Tb usable disk space, works well, no gotchas?
- make it visible to both proxVE hosts,
- VM LVMs will be carved off in here by proxmox as disk space is allocated for VMs using this storage pool.
or
- is there a maximum size for the LVM Volume (ie, don't go above 12 Tb per LUN? deploy as 2 x 12Tb instead of 1 x 24Tb ?)
- is there any inherent performance limit which makes LVM get stupid or more PITA to deal with above a certain size ?
I know LVM on 64bit platform supports 'really big' disk capacity in theory. I also know from experience (for example) that ext4 on 64bit is 'meant' to support really big disk capacity but for some platforms, for some support libraries for e2fs utils it means you can't format really big volumes (ie, >12 Tb for example) without having to either compile yourself properly from scratch the disk formatting tools, or other workarounds. So I figure I might as well ask if anyone has strong recommendatoin around this kind of config where I have the choice of deploying a single 24Tb storage volume 'for simplicity' or better to do other things (2x12 Tb, 3 x 8Tb?) for example.
Any comments are greatly appreciated.
Thanks!
Tim Chipman