Quantcast
Channel: Proxmox Support Forum
Viewing all articles
Browse latest Browse all 170561

same mac from same proxmox node appears multiple times in switch mac list?

$
0
0
Hi all,

I have some network issues.. i think its because of loops but both RSTP and loop protection is active ..still i may have done it wrong.. anyway.. im asking here cause i need to know what proxmox normal behaviour is..

So i have 3 nodes with two interfaces in bond0 with lacp configured on each node - pretty standard..

I have two lan switches which each interface from each node is connected to. These two switches are also trunked with LACP.. I have tried with both HP static trunk and LACP - made no difference..

what im seeing aka symptoms..

sometimes when i boot a node or restart a node network service it all of sudden start to drop packets. and after some reboots i stops again. I quicker way to solve it is to boot one of the switches. My theroy is the following:

LACP makes only one interface active.. and im thinking in the packet drop scenario that 2 nodes has thire interface active on switch 1 and the last node has his interface active on switch 2 which in turn, because of the trunk between the two switches creates a loop.. does this make sense?

I went to the proxmox nodes and did a ifconfig to find the MACs.. here i noticed ALL my interfaces had the same mac

Proxmox00
Code:

root@proxmox00:~# ifconfig
bond0    Link encap:Ethernet  HWaddr 80:c1:6e:64:8d:3c
          UP BROADCAST RUNNING MASTER MULTICAST  MTU:1500  Metric:1
          RX packets:2712897 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1195000 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:1835251437 (1.7 GiB)  TX bytes:238454039 (227.4 MiB)

bond0.2  Link encap:Ethernet  HWaddr 80:c1:6e:64:8d:3c
          UP BROADCAST RUNNING MASTER MULTICAST  MTU:1500  Metric:1
          RX packets:1023654 errors:0 dropped:0 overruns:0 frame:0
          TX packets:754 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:1282156890 (1.1 GiB)  TX bytes:49772 (48.6 KiB)

bond0.3  Link encap:Ethernet  HWaddr 80:c1:6e:64:8d:3c
          UP BROADCAST RUNNING MASTER MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:754 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:0 (0.0 B)  TX bytes:49772 (48.6 KiB)

eth2      Link encap:Ethernet  HWaddr 80:c1:6e:64:8d:3c
          UP BROADCAST RUNNING SLAVE MULTICAST  MTU:9000  Metric:1
          RX packets:2210271 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1148199 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:1583448270 (1.4 GiB)  TX bytes:232651227 (221.8 MiB)

eth3      Link encap:Ethernet  HWaddr 80:c1:6e:64:8d:3c
          UP BROADCAST RUNNING SLAVE MULTICAST  MTU:9000  Metric:1
          RX packets:502626 errors:0 dropped:0 overruns:0 frame:0
          TX packets:46801 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:251803167 (240.1 MiB)  TX bytes:5802812 (5.5 MiB)

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:94 errors:0 dropped:0 overruns:0 frame:0
          TX packets:94 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:16339 (15.9 KiB)  TX bytes:16339 (15.9 KiB)

venet0    Link encap:UNSPEC  HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00                    -00
          inet6 addr: fe80::1/128 Scope:Link
          UP BROADCAST POINTOPOINT RUNNING NOARP  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:3 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

vmbr0    Link encap:Ethernet  HWaddr 80:c1:6e:64:8d:3c
          inet addr:10.10.99.20  Bcast:10.10.99.255  Mask:255.255.255.0
          inet6 addr: fe80::82c1:6eff:fe64:8d3c/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:1125198 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1094891 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:264351866 (252.1 MiB)  TX bytes:226508308 (216.0 MiB)

vmbr1    Link encap:Ethernet  HWaddr 80:c1:6e:64:8d:3c
          inet6 addr: fe80::82c1:6eff:fe64:8d3c/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:1272 errors:0 dropped:0 overruns:0 frame:0
          TX packets:7 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:58512 (57.1 KiB)  TX bytes:578 (578.0 B)

vmbr3    Link encap:Ethernet  HWaddr 80:c1:6e:64:8d:3c
          inet6 addr: fe80::82c1:6eff:fe64:8d3c/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:7 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:0 (0.0 B)  TX bytes:578 (578.0 B)

root@proxmox00:~#

Proxmox01
Code:

root@proxmox01:~# ifconfig
bond0    Link encap:Ethernet  HWaddr e8:39:35:b7:c7:4c
          UP BROADCAST RUNNING MASTER MULTICAST  MTU:1500  Metric:1
          RX packets:78985596 errors:0 dropped:0 overruns:0 frame:0
          TX packets:101958145 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:47848136947 (44.5 GiB)  TX bytes:103293213480 (96.1 GiB)

bond0.2  Link encap:Ethernet  HWaddr e8:39:35:b7:c7:4c
          UP BROADCAST RUNNING MASTER MULTICAST  MTU:1500  Metric:1
          RX packets:11528034 errors:0 dropped:0 overruns:0 frame:0
          TX packets:7606523 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:9308896784 (8.6 GiB)  TX bytes:13112974005 (12.2 GiB)

eth2      Link encap:Ethernet  HWaddr e8:39:35:b7:c7:4c
          UP BROADCAST RUNNING SLAVE MULTICAST  MTU:9000  Metric:1
          RX packets:11207054 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1364502 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:7576213855 (7.0 GiB)  TX bytes:202885911 (193.4 MiB)

eth3      Link encap:Ethernet  HWaddr e8:39:35:b7:c7:4c
          UP BROADCAST RUNNING SLAVE MULTICAST  MTU:9000  Metric:1
          RX packets:67778542 errors:0 dropped:0 overruns:0 frame:0
          TX packets:100593643 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:40271923092 (37.5 GiB)  TX bytes:103090327569 (96.0 GiB)

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:1208712 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1208712 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:11790128367 (10.9 GiB)  TX bytes:11790128367 (10.9 GiB)

tap101i0  Link encap:Ethernet  HWaddr c2:15:aa:51:80:23
          inet6 addr: fe80::c015:aaff:fe51:8023/64 Scope:Link
          UP BROADCAST RUNNING PROMISC MULTICAST  MTU:1500  Metric:1
          RX packets:1917199 errors:0 dropped:0 overruns:0 frame:0
          TX packets:4189948 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:500
          RX bytes:121642134 (116.0 MiB)  TX bytes:5335729264 (4.9 GiB)

tap102i0  Link encap:Ethernet  HWaddr 92:76:1b:e4:58:1f
          inet6 addr: fe80::9076:1bff:fee4:581f/64 Scope:Link
          UP BROADCAST RUNNING PROMISC MULTICAST  MTU:1500  Metric:1
          RX packets:5 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:1 carrier:0
          collisions:0 txqueuelen:500
          RX bytes:300 (300.0 B)  TX bytes:0 (0.0 B)

tap112i0  Link encap:Ethernet  HWaddr 1a:b3:20:e5:5c:7b
          inet6 addr: fe80::18b3:20ff:fee5:5c7b/64 Scope:Link
          UP BROADCAST RUNNING PROMISC MULTICAST  MTU:1500  Metric:1
          RX packets:5 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:1 carrier:0
          collisions:0 txqueuelen:500
          RX bytes:300 (300.0 B)  TX bytes:0 (0.0 B)

tap113i0  Link encap:Ethernet  HWaddr a6:7b:dc:97:43:16
          inet6 addr: fe80::a47b:dcff:fe97:4316/64 Scope:Link
          UP BROADCAST RUNNING PROMISC MULTICAST  MTU:1500  Metric:1
          RX packets:420012 errors:0 dropped:0 overruns:0 frame:0
          TX packets:18693350 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:500
          RX bytes:219791105 (209.6 MiB)  TX bytes:22361016498 (20.8 GiB)

venet0    Link encap:UNSPEC  HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00                                  -00
          inet6 addr: fe80::1/128 Scope:Link
          UP BROADCAST POINTOPOINT RUNNING NOARP  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:3 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

vmbr0    Link encap:Ethernet  HWaddr e8:39:35:b7:c7:4c
          inet addr:10.10.99.21  Bcast:10.10.99.255  Mask:255.255.255.0
          inet6 addr: fe80::ea39:35ff:feb7:c74c/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:43488538 errors:0 dropped:0 overruns:0 frame:0
          TX packets:39379002 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:32323093880 (30.1 GiB)  TX bytes:86505250883 (80.5 GiB)

vmbr1    Link encap:Ethernet  HWaddr e8:39:35:b7:c7:4c
          inet6 addr: fe80::ea39:35ff:feb7:c74c/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:28063 errors:0 dropped:0 overruns:0 frame:0
          TX packets:7 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:1211608 (1.1 MiB)  TX bytes:578 (578.0 B)

root@proxmox01:~#

Proxmox02
Code:

root@proxmox02:~# ifconfig
bond0    Link encap:Ethernet  HWaddr 80:c1:6e:64:ab:2a
          UP BROADCAST RUNNING MASTER MULTICAST  MTU:1500  Metric:1
          RX packets:44492827 errors:0 dropped:0 overruns:0 frame:0
          TX packets:27420072 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:14991834618 (13.9 GiB)  TX bytes:4781766759 (4.4 GiB)

bond0.2  Link encap:Ethernet  HWaddr 80:c1:6e:64:ab:2a
          UP BROADCAST RUNNING MASTER MULTICAST  MTU:1500  Metric:1
          RX packets:6646040 errors:0 dropped:0 overruns:0 frame:0
          TX packets:15976 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:3963772204 (3.6 GiB)  TX bytes:1054424 (1.0 MiB)

eth2      Link encap:Ethernet  HWaddr 80:c1:6e:64:ab:2a
          UP BROADCAST RUNNING SLAVE MULTICAST  MTU:9000  Metric:1
          RX packets:15261688 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1036273 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:8401169881 (7.8 GiB)  TX bytes:131043042 (124.9 MiB)

eth3      Link encap:Ethernet  HWaddr 80:c1:6e:64:ab:2a
          UP BROADCAST RUNNING SLAVE MULTICAST  MTU:9000  Metric:1
          RX packets:29231139 errors:0 dropped:0 overruns:0 frame:0
          TX packets:26383799 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:6590664737 (6.1 GiB)  TX bytes:4650723717 (4.3 GiB)

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:1158 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1158 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:657942 (642.5 KiB)  TX bytes:657942 (642.5 KiB)

venet0    Link encap:UNSPEC  HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
          inet6 addr: fe80::1/128 Scope:Link
          UP BROADCAST POINTOPOINT RUNNING NOARP  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:3 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

vmbr0    Link encap:Ethernet  HWaddr 80:c1:6e:64:ab:2a
          inet addr:10.10.99.22  Bcast:10.10.99.255  Mask:255.255.255.0
          inet6 addr: fe80::82c1:6eff:fe64:ab2a/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:27918154 errors:0 dropped:0 overruns:0 frame:0
          TX packets:25346691 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:6052590170 (5.6 GiB)  TX bytes:4530105646 (4.2 GiB)

vmbr1    Link encap:Ethernet  HWaddr 80:c1:6e:64:ab:2a
          inet6 addr: fe80::82c1:6eff:fe64:ab2a/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:27945 errors:0 dropped:0 overruns:0 frame:0
          TX packets:7 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:1285576 (1.2 MiB)  TX bytes:578 (578.0 B)

root@proxmox02:~# ^C

I figure its because all my interfaces comes in one way or another from bond0 but still eth2 and eth3 on proxmox00 also has the same MAC - is this suppose to be that way?

LACP on the switch trunk side is configured in the following groups:

TRK1: switch trunk
TRK2: Storage NAS - not important
TRK3: BAckup Nas - not important
TRK4: Proxmox00
TRK5: Proxmox01
TRK6: Proxmox02

Here is the mac table of switch 1

Code:


MAC Address

Source Port

MAC Type

00:11:32:24:28:9b TRK2 Learned
00:11:32:24:28:d7 TRK3 Learned
00:11:32:24:28:da TRK1 Learned
00:90:fb:40:9c:f6 TRK1 Learned
00:e0:20:11:0a:2e 12 Learned
00:e0:20:11:0a:2e 12 Learned
72:e1:e0:41:93:73 TRK5 Learned
80:c1:6e:64:8d:3c TRK1 Learned
80:c1:6e:64:8d:3c TRK1 Learned
80:c1:6e:64:8d:3d TRK4 Learned
80:c1:6e:64:ab:2a TRK6 Learned
80:c1:6e:64:ab:2a TRK6 Learned
80:c1:6e:64:ab:2b TRK6 Learned
c2:31:d6:cc:cf:4b TRK5 Learned
d4:c9:ef:3a:fe:e0 TRK1 Learned
d4:c9:ef:3a:fe:f8 TRK1 Learned
d4:c9:ef:3c:d9:a0 CPU Management
e8:39:35:b7:c7:4c TRK5 Learned
e8:39:35:b7:c7:4c TRK5 Learned
e8:39:35:b7:c7:4d TRK5 Learned

Notice there are 3 entries for TRK6 and that the mac is the same for two of them and not all 3?

also i see allot of double mac entries for same trunk groups.. actually i expected to only see one MAC for each trunk group? is that normal proxmox behavior or could it be because of switch loops? Would a normal setup with a proxmox node connected to two switche using LACP look this way or?

THANKS

Casper

EDIT:

Here is mac table of switch 2

Code:


MAC Address

Source Port

MAC Type

00:11:32:24:28:9b TRK1 Learned
00:11:32:24:28:9c TRK2 Learned
00:11:32:24:28:d7 TRK3 Learned
00:11:32:24:28:d8 TRK3 Learned
00:11:32:24:28:da 22 Learned
00:90:fb:40:9c:f6 12 Learned
00:e0:20:11:0a:2e TRK1 Learned
00:e0:20:11:0a:2e TRK1 Learned
72:e1:e0:41:93:73 TRK1 Learned
80:c1:6e:64:8d:3c TRK4 Learned
80:c1:6e:64:8d:3c TRK4 Learned
80:c1:6e:64:ab:2a TRK1 Learned
80:c1:6e:64:ab:2a TRK1 Learned
d4:c9:ef:3a:fe:e0 CPU Management
d4:c9:ef:3c:d9:b8 TRK1 Learned
e8:39:35:b7:c7:4c TRK5 Learned
e8:39:35:b7:c7:4c TRK1 Learned


Viewing all articles
Browse latest Browse all 170561

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>