Sunday, September 28, 2014

Demystifying VLT

What is VLT?

VLT means virtual link trunking.  It is a proprietary Force10 technology.  It does the same things as Cisco's Virtual Port Channel (VPC) and IEEE 802.1AX-2008 multi-chassis LAG (MLAG).  The idea is the exact same.  You create a set of VLT peers.  Once VLT is up and running, you can take another switch and create a LAG split between those two VLT peers.

VLT peering causes the two switches to become a single virtual switch so that, say, another switch which splits a LAG between the two VLT peers doesn't realize that it's splitting its LAG between two switches.  It thinks it's connected to one switch.  VLT can do this by peering switches together and by syncing ARP and MAC entries between the switches.  So in the simplest VLT scenario like depicted below, only the two switches at the top have VLT configuration and can be said to be participating in VLT.  The switch at the bottom has a single ordinary LAG connecting to the two switches at the top.  It does, however, have to match the type of LAG that the VLT switches are using--LACP or static.
This is the simplest VLT scenario possible.


















What problems does VLT solve?

I think of VLT as an in-between solution--somewhere between stacking, and not stacking.  So for instance, with VLT you have link redundancy, with no blocking and full link utilization.  But you can get the same things by simply stacking the switches.  So why don't people just stack the switches to effectively make them one switch instead of bothering with VLT?  Good question, I'm glad you asked.  Because switches in VLT are not true stacks they have some level of independence which makes upgrades relatively painless.  It may be possible to upgrade a switch stack without any downtime, but it's a terrible idea to try to do.  But you can do it with VLT with some caveats.**
Upgrading a stack is a nightmare for SLA-critical environments






















The non-stacking solution involves the use of STP.



















This is a fairly common setup.  This solution provides switch independence which makes upgrades easier.  However, because there is a loop, STP must block a port.  This often causes issues because it's common for people to not tune STP parameters so that the proper port gets blocked.  You generally do not want the interswitch LAG to be blocking.  The problem with a blocked link, of course, is the loss of bandwidth.  You have an entire link that is in standby, which in this scenario, is likely to be a 40gbps port, since s4820t switches have 40gbps uplinks.  That's a massive amount of wasted bandwidth.  But there are hacky ways around this, such as the implementation of PVST, or MSTP.  By configuring PVST or MSTP, you can avoid wasting bandwidth by selecting blocked ports on a per VLAN basis, but it's a kludge.  There's more configuration and consideration involved.  People are unlikely to implement it properly.  VLT allows you to avoid all of that.

So in essence what VLT does is connect switches so that they can appear to be one switch from the outside of the VLT domain in order to allow for loopless redundancy, while having a level of independence between one another for easy upgrades.


Configuration example

These are the steps involved in getting it all set up for this particular scenario.  It's not a one-size fits all.  I got this configuration sample here.  I omit parts of the configuration, because there's lot of redundancy.  But I explain everything pretty completely about which parts are missing.

  • Configure RSTP.  One peer should be 0; other should be 4096.
    This is to ensure that the STP root is one of the VLT peers.
  • Check and open ports.
    Make sure that the ports are enabled.  FTOS has ports disabled by default.
  • Configure backup link.
    This generally uses management ports to maintain the health of the VLT peer status.
  • Configure VLTi port-channel.
    This is used to synchronize ARP and MAC information across between the peers so that they can appear to be one switch from the outside.  
  • Configure VLT domain (vlt domain 999, back-up destination, peer link port-channel)
  • Verify VLT domain and VLTi are working
    Sh vlt brief
    This is how you can know that the VLT is working.
  • Configure access port-channel on peer
    Here you configure the port channels are on each peer and relate them to one another. 
  • Create normal port channel, then specify that it is part of peer LAG (vlt-peer-lag)
    This refers to the port-channel on the switch outside of the VLT domain.




















Force10_VLTPeer1 Switch Configuration
Force10(conf)#hostname Force10_VLTPeer1

1. Configure RSTP
Force10_VLTPeer1#configure
Force10_VLTPeer1(conf)#protocol spanning-tree rstp
Force10_VLTPeer1(conf-rstp)#no disable
Force10_VLTPeer1(conf-rstp)#00:04:08: %STKUNIT1-M:CP %SPANMGR-5-STP_ROOT_CHANGE: RSTP root changed. My Bridge ID: 32768:0001.e88b.1b1d Old Root: 32768:0000.0000.0000 New Root: 32768:0001.e88b.1b1d
Force10_VLTPeer1(conf-rstp)#bridge-priority 0
Force10_VLTPeer1(conf-rstp)#00:04:17: %STKUNIT1-M:CP %SPANMGR-5-STP_ROOT_CHANGE: RSTP root changed. My Bridge ID: 0:0001.e88b.1b1d Old Root: 32768:0001.e88b.1b1d New Root: 0:0001.e88b.1b1d
Force10_VLTPeer1(conf-rstp)#end

This sets up RSTP on Peer 1, and configures it to be the root. You'll be doing the same on Peer 2, except you'll be using priority 4096.  

2. Check and open ports
Force10_VLTPeer1#show interfaces status | grep 24|60
Te 1/24               Down   Auto      Auto   --
Fo 1/60               Down   40000 Mbit Auto   --
Force10_VLTPeer1#configure
Force10_VLTPeer1(conf)#interface range te 1/24 , fo 1/60
Force10_VLTPeer1(conf-if-range-te-1/24,fo-1/60)#no shutdown
00:05:57: %STKUNIT1-M:CP %IFMGR-5-ASTATE_UP: Changed interface Admin state to up: Te 1/24
00:05:57: %STKUNIT1-M:CP %IFMGR-5-ASTATE_UP: Changed interface Admin state to up: Fo 1/60
Force10_VLTPeer1(conf-if-range-te-1/24,fo-1/60)#end

Here, you're just making sure that the ports are open.  You'll do it on both switches.

3a. Configure VLTI port-channel
Force10_VLTPeer1(conf)#interface port-channel 100
Force10_VLTPeer1(conf-if-po-100)#no switchport
Force10_VLTPeer1(conf-if-po-100)#no ip address
Force10_VLTPeer1(conf-if-po-100)#channel-member fortyGigE 1/60
Force10_VLTPeer1(conf-if-po-100)#no shutdown
00:07:00: %STKUNIT1-M:CP %IFMGR-5-ASTATE_UP: Changed interface Admin state to up: Po 100
Force10_VLTPeer1(conf-if-po-100)#end

This creates the LAG which is going to be used for the VLT Interconnect (VLTi).  You need to do it on both peers.

3b. Configure VLT Domain
Force10_VLTPeer2#sh ip interface brief | grep YES
ManagementEthernet 0/0   172.28.17.251   YES Manual up                    up
Force10_VLTPeer1#configure
Force10_VLTPeer1(conf)#vlt domain 999
Force10_VLTPeer1(conf-vlt-domain)#back-up destination 172.28.17.251
Force10_VLTPeer1(conf-vlt-domain)#peer-link port-channel 100
Force10_VLTPeer1(conf-vlt-domain)#end

This sets up the VLT domain.  It configures the domain ID (999), it configures the backup link (for heartbeat and health of VLTi), and sets the previously created port-channel to be the VLTi.  You'll do the same thing on the other peer, but you'll configure the back-up to be the management IP of the other switch.  Otherwise, it'll all be the same.

3c. Verify VLT Domain and VLTI (ICL) are working
(when Peer 2 configuration is completed)
Force10_VLTPeer1#show vlt brief
 VLT Domain Brief
------------------
 Domain ID:                    999
 Role:                         Secondary
 Role Priority:                32768
 ICL Link Status:              Up
 HeartBeat Status:             Up
 VLT Peer Status:              Up
 Local System MAC address:     00:01:e8:8b:1b:1d
 Remote System MAC address:    00:01:e8:8b:19:ac
Force10_VLTPeer1#

'show vlt brief' will tell you whether your VLT is functional.  

4. Configure access port-channel on peer
Force10_VLTPeer1#configure
Force10_VLTPeer1(conf)#interface tengigabitethernet 1/24
Force10_VLTPeer1(conf-if-te-1/24)#port-channel-protocol lacp
Force10_VLTPeer1(conf-if-te-1/24-lacp)#port-channel 110 mode active
00:22:42: %STKUNIT1-M:CP %IFMGR-5-ASTATE_UP: Changed interface Admin state to up: Po 110
Force10_VLTPeer1(conf-if-te-1/24-lacp)#exit
Force10_VLTPeer1(conf-if-te-1/24)#exit
Force10_VLTPeer1(conf)#interface port-channel 110
Force10_VLTPeer1(conf-if-po-110)#switchport
Force10_VLTPeer1(conf-if-po-110)#vlt-peer-lag port-channel 110
Force10_VLTPeer1(conf-if-po-110)#end

This sets up the access port-channel--the port-channel which is going to connect to the s50 which is not part of the VLT domain.  Notice that he sets it up like an ordinary LACP port-channel, but under the port-channel, after he configures it to be in switchport mode, he mentions that it's part of the peer LAG.  The same is done on the other peer.  

Access Switch Configuration
Force10#show interfaces status | grep 45|47
Gi 1/45               Down   Auto      Auto   --
Gi 1/47               Down   Auto      Auto   --
Force10#configure
Force10(conf)#interface range gi 1/45 , gi 1/47
Force10(conf-if-range-gi-1/45,gi-1/47)#no shutdown
02:02:46: %STKUNIT1-M:CP %IFMGR-5-ASTATE_UP: Changed interface Admin state to up: Gi 1/45
02:02:46: %STKUNIT1-M:CP %IFMGR-5-ASTATE_UP: Changed interface Admin state to up: Gi 1/47
Force10(conf-if-range-gi-1/45,gi-1/47)#02:02:46: %STKUNIT1-M:CP %IFMGR-5-OSTATE_UP: Changed interface state to up: Gi 1/45
02:02:46: %STKUNIT1-M:CP %IFMGR-5-OSTATE_UP: Changed interface state to up: Gi 1/47
Force10(conf-if-range-gi-1/45,gi-1/47)#port-channel-protocol lacp
Force10(conf-if-range-gi-1/45,gi-1/47-lacp)#port-channel 110 mode active
02:04:19: %STKUNIT1-M:CP %IFMGR-5-ASTATE_UP: Changed interface Admin state to up: Po 110
Force10(conf-if-range-gi-1/45,gi-1/47-lacp)#02:04:21: %STKUNIT1-M:CP %LACP-5-PORT-GROUPED: PortChannel-110-Grouped: Interface Gi 1/45 joined port-channel 110
02:04:21: %STKUNIT1-M:CP %IFMGR-5-OSTATE_UP: Changed interface state to up: Po 110
02:04:22: %STKUNIT1-M:CP %LACP-5-PORT-GROUPED: PortChannel-110-Grouped: Interface Gi 1/47 joined port-channel 110

Configuration on this switch is simple.  You just set up the port-channel as though you're connecting to a single switch.  


Building on VLT

Once you've created the VLT domain and VLTi, if you want to make a more complex topology, all you need to do is create more access port-channels.  This is simple.  You just repeat step four but with the additional access port-channels, po111, and po112 on each peer.  Then you set up the port-channels on the switches which are on the other end of the access port-channels.





















Other considerations
This is a tutorial to explain the basics of VLT.  There are many different kinds of VLT topologies shown in the Force10 Configuration Guide.

You might wonder why use VLT at all.  You could achieve many of the same results by using OSPF.  This is possible because OSPF does equal cost load balancing.  By using OSPF for load balancing you can have it all: full utilization of all links and switch independence for easy upgrades.  Indeed, that's why Force10 uses OSPF for their spine and leaf architecture.  The only problem is that it doesn't allow you to extend layer 2 out to the core.  So that makes it hard for devices on a common network to communicate with each other if they're on different switches.  So the type of non-blocking architecture that you employ really depends on traffic flows your network uses.

PC1 and PC2 cannot be in the same
network if OSPF is being used for non-
blocking architecture.
















Caveats

  • Don't try to implement a VLT architecture in a live production network unless really know what you're doing.  
  • When implementing VLT, use a configuration that you know works.  VLT has a lot of dependencies and caveats.  
  • Read the manual for caveats.  There is a giant list of them (use RSTP, don't do VLT with stacked switches, etc.).  VLT is not as simple as setting up a LAG, and it needs to be set up properly, and tested in order for it to work well.  Don't just think, 'Well, I understand the theory, and this should work.'  Don't implement VLT willy nilly.  


**Because VLT provides some switch independence, having VLT pairs running different firmware versions should not be an issue.  But I've run into issues with ARP entries not syncing across the VLT peers if they're not running the same firmware.  It's all documented in the release notes.  So much for seamless upgrades.  That said, it's still much better than upgrading stacked switches.  Bottom line: schedule a maintenance period for upgrades.