交换机上启用LLDP、PFC和DCBx,PFC优先级为3(默认值),DCBx的标准为IEEE(默认值)。
dcb pfc
#
interface 25GE1/0/10
description Host-01
port link-type trunk
undo port trunk allow-pass vlan 1
port trunk allow-pass vlan 125
stp edged-port enable
lldp tlv-enable dcbx
dcb pfc enable mode manual
#
interface 25GE1/0/12
description Host-02
port link-type trunk
undo port trunk allow-pass vlan 1
port trunk allow-pass vlan 125
stp edged-port enable
lldp tlv-enable dcbx
dcb pfc enable mode manual
#
lldp enable
#
通过主机-配置-物理适配器确认用于vSAN网卡的设备位置和端口,以下示例中是0000:8a:00的端口2(vmnic3)和0000:8b:00的端口1(vmnic4),首先查看一下LLDP和DCBx的相关配置。需要先安装MFT才能使用相关命令。
[root@yaoge123:~] /opt/mellanox/bin/mlxconfig -d 0000:8a:00.1 query|grep -iE "dcb|lldp"
LLDP_NB_DCBX_P1 False(0)
LLDP_NB_RX_MODE_P1 OFF(0)
LLDP_NB_TX_MODE_P1 OFF(0)
LLDP_NB_DCBX_P2 False(0)
LLDP_NB_RX_MODE_P2 OFF(0)
LLDP_NB_TX_MODE_P2 OFF(0)
DCBX_IEEE_P1 True(1)
DCBX_CEE_P1 True(1)
DCBX_WILLING_P1 True(1)
DCBX_IEEE_P2 True(1)
DCBX_CEE_P2 True(1)
DCBX_WILLING_P2 True(1)
[root@yaoge123:~] /opt/mellanox/bin/mlxconfig -d 0000:8b:00.0 query|grep -iE "dcb|lldp"
LLDP_NB_DCBX_P1 False(0)
LLDP_NB_RX_MODE_P1 OFF(0)
LLDP_NB_TX_MODE_P1 OFF(0)
LLDP_NB_DCBX_P2 False(0)
LLDP_NB_RX_MODE_P2 OFF(0)
LLDP_NB_TX_MODE_P2 OFF(0)
DCBX_IEEE_P1 True(1)
DCBX_CEE_P1 True(1)
DCBX_WILLING_P1 True(1)
DCBX_IEEE_P2 True(1)
DCBX_CEE_P2 True(1)
DCBX_WILLING_P2 True(1)
发现LLDP没有启动,启用LLDP和DCBx,DCBx IEEE已启用。
[root@yaoge123:~] /opt/mellanox/bin/mlxconfig -d 0000:8a:00.1 set LLDP_NB_DCBX_P2=1 LLDP_NB_RX_MODE_P2=2 LLDP_NB_TX_MODE_P2=2 DCBX_WILLING_P2=1 DCBX_IEEE_P2=1
Device #1:
----------
Device type: ConnectX5
Name: MCX512A-ACU_Ax_Bx
Description: ConnectX-5 EN network interface card; 10/25GbE dual-port SFP28; PCIe3.0 x8; UEFI Enabled (x86/ARM)
Device: 0000:8a:00.1
Configurations: Next Boot New
LLDP_NB_DCBX_P2 False(0) True(1)
LLDP_NB_RX_MODE_P2 OFF(0) ALL(2)
LLDP_NB_TX_MODE_P2 OFF(0) ALL(2)
DCBX_WILLING_P2 True(1) True(1)
DCBX_IEEE_P2 True(1) True(1)
Apply new Configuration? (y/n) [n] : y
Applying... Done!
-I- Please reboot machine to load new configurations.
[root@yaoge123:~] /opt/mellanox/bin/mlxconfig -d 0000:8b:00.0 set LLDP_NB_DCBX_P1=1 LLDP_NB_RX_MODE_P1=2 LLDP_NB_TX_MODE_P1=2 DCBX_WILLING_P1=1 DCBX_IEEE_P1=1
Device #1:
----------
Device type: ConnectX5
Name: MCX512A-ACU_Ax_Bx
Description: ConnectX-5 EN network interface card; 10/25GbE dual-port SFP28; PCIe3.0 x8; UEFI Enabled (x86/ARM)
Device: 0000:8b:00.0
Configurations: Next Boot New
LLDP_NB_DCBX_P1 False(0) True(1)
LLDP_NB_RX_MODE_P1 OFF(0) ALL(2)
LLDP_NB_TX_MODE_P1 OFF(0) ALL(2)
DCBX_WILLING_P1 True(1) True(1)
DCBX_IEEE_P1 True(1) True(1)
Apply new Configuration? (y/n) [n] : y
Applying... Done!
-I- Please reboot machine to load new configurations.
重启后查看网卡当前配置,确认LLDP和DCBx均已启用
[root@yaoge123:~] /opt/mellanox/bin/mlxconfig -d 0000:8a:00.1 query|grep -iE "dcb|lldp"
LLDP_NB_DCBX_P1 False(0)
LLDP_NB_RX_MODE_P1 OFF(0)
LLDP_NB_TX_MODE_P1 OFF(0)
LLDP_NB_DCBX_P2 True(1)
LLDP_NB_RX_MODE_P2 ALL(2)
LLDP_NB_TX_MODE_P2 ALL(2)
DCBX_IEEE_P1 True(1)
DCBX_CEE_P1 True(1)
DCBX_WILLING_P1 True(1)
DCBX_IEEE_P2 True(1)
DCBX_CEE_P2 True(1)
DCBX_WILLING_P2 True(1)
[root@yaoge123:~] /opt/mellanox/bin/mlxconfig -d 0000:8b:00.0 query|grep -iE "dcb|lldp"
LLDP_NB_DCBX_P1 True(1)
LLDP_NB_RX_MODE_P1 ALL(2)
LLDP_NB_TX_MODE_P1 ALL(2)
LLDP_NB_DCBX_P2 False(0)
LLDP_NB_RX_MODE_P2 OFF(0)
LLDP_NB_TX_MODE_P2 OFF(0)
DCBX_IEEE_P1 True(1)
DCBX_CEE_P1 True(1)
DCBX_WILLING_P1 True(1)
DCBX_IEEE_P2 True(1)
DCBX_CEE_P2 True(1)
DCBX_WILLING_P2 True(1)
查看网卡的DCB状态,模式为IEEE,PFC已启用且优先级为3(0 0 0 1 0 0 0 0)
[root@2288Hv6-05:~] esxcli network nic dcb status get -n vmnic3
Nic Name: vmnic3
Mode: 3 - IEEE Mode
Enabled: true
Capabilities:
Priority Group: true
Priority Flow Control: true
PG Traffic Classes: 8
PFC Traffic Classes: 8
PFC Enabled: true
PFC Configuration: 0 0 0 1 0 0 0 0
IEEE ETS Configuration:
Willing Bit In ETS Config TLV: 1
Supported Capacity: 8
Credit Based Shaper ETS Algorithm Supported: 0x0
TX Bandwidth Per TC: 13 13 13 13 12 12 12 12
RX Bandwidth Per TC: 13 13 13 13 12 12 12 12
TSA Assignment Table Per TC: 2 2 2 2 2 2 2 2
Priority Assignment Per TC: 1 0 2 3 4 5 6 7
Recommended TC Bandwidth Per TC: 13 13 13 13 12 12 12 12
Recommended TSA Assignment Per TC: 2 2 2 2 2 2 2 2
Recommended Priority Assignment Per TC: 1 0 2 3 4 5 6 7
IEEE PFC Configuration:
Number Of Traffic Classes: 8
PFC Configuration: 0 0 0 1 0 0 0 0
Macsec Bypass Capability Is Enabled: 0
Round Trip Propagation Delay Of Link: 0
Sent PFC Frames: 0 0 0 0 0 0 0 0
Received PFC Frames: 0 0 0 0 0 0 0 0
DCB Apps:
[root@2288Hv6-05:~] esxcli network nic dcb status get -n vmnic4
Nic Name: vmnic4
Mode: 3 - IEEE Mode
Enabled: true
Capabilities:
Priority Group: true
Priority Flow Control: true
PG Traffic Classes: 8
PFC Traffic Classes: 8
PFC Enabled: true
PFC Configuration: 0 0 0 1 0 0 0 0
IEEE ETS Configuration:
Willing Bit In ETS Config TLV: 1
Supported Capacity: 8
Credit Based Shaper ETS Algorithm Supported: 0x0
TX Bandwidth Per TC: 13 13 13 13 12 12 12 12
RX Bandwidth Per TC: 13 13 13 13 12 12 12 12
TSA Assignment Table Per TC: 2 2 2 2 2 2 2 2
Priority Assignment Per TC: 1 0 2 3 4 5 6 7
Recommended TC Bandwidth Per TC: 13 13 13 13 12 12 12 12
Recommended TSA Assignment Per TC: 2 2 2 2 2 2 2 2
Recommended Priority Assignment Per TC: 1 0 2 3 4 5 6 7
IEEE PFC Configuration:
Number Of Traffic Classes: 8
PFC Configuration: 0 0 0 1 0 0 0 0
Macsec Bypass Capability Is Enabled: 0
Round Trip Propagation Delay Of Link: 0
Sent PFC Frames: 0 0 0 0 0 0 0 0
Received PFC Frames: 0 0 0 0 0 0 0 0
DCB Apps: