ArubaOS CX Virtual Switching Extension – VSX Stacking Guide

What is VSX?

VSX is a cluster technology that allows the two VSX switches to run with independent control planes (OSPF/BGP) and present themselves as different routers in the network. In the datapath, however, they function as a single router and support active-active forwarding.

VSX allows you to mitigate inherent issues with a shared control plane that comes with traditional stacking while maintaining all the benefits

  • Control plane: Inter-Switch-Link and Keepalive
  • Data plane L2: MCLAGs
  • Data plane L3: Active gateway

This is a very similar technology compared to Dell VLT stacking with Dell OS10

Basic feature Comparison with Dell VLT Stacking

Dell VLT StackingAruba VSX
Supports Multi chassis Lag
independent control planes
All active-gateway configuration (L3 load balancing)✅(VLT Peer routing)(VSX Active forwarding)
Layer 3 Packet load balancing
Can Participate in Spanning tree MST/RSTP
Gateway IP Redundancy ✅VRRP✅(VSX Active Gateway or VRRP)

Setup Guide

What you need?

  • 10/25/40/100GE Port for the interswitch link
  • VSX supported switch, VSX is only supported on switches above CX6300 SKU
Switch SeriesVSX
CX 6200 seriesX
CX 6300 seriesX
CX 6400 series
CX 8200 series
CX 8320/8325 series
CX 8360 series
**Updated 2020-Dec

For this guide im using a 8325 series switch

Dry run

  • Setup LAG interface for the inter-switch link (ISL)
  • Create the VSX cluster
  • Setup a keepalive link and a new VRF for the keepalive traffic

Setup LAG interface for the inter-switch link (ISL)

In order to form the VSX cluster, we need a LAG interface for the inter switch communication

Naturally i pick the fastest ports on the switch to create this 10/25/40/100GE

Depending on what switch you have, The ISL bandwidth can be a limitation/Bottleneck, Account for this factor when designing a VSX based solution 
Utilize VSX-Activeforwarding or Active gateways to mitigate this

Create the LAG interface

This is a regular Port channel no special configurations, you need to create this on both switches

  • Native VLAN needs to be the default VLAN
  • Trunk port and All VLANs allowed
CORE01#

interface lag 256
no shutdown
description VSX-LAG
no routing
vlan trunk native 1 tag
vlan trunk allowed all
lacp mode active
exit


-------------------------------

CORE02#

interface lag 256
no shutdown
description VSX-LAG
no routing
vlan trunk native 1 tag
vlan trunk allowed all
lacp mode active
exit
Add/Assign the physical ports to the LAG interface

I’m using two 100GE ports for the ISL LAG

CORE01#

interface 1/1/55
no shutdown
lag 256
exit
interface 1/1/56
no shutdown
lag 256
exit

-------------------------------

CORE02#

interface 1/1/55
no shutdown
lag 256
exit
interface 1/1/56
no shutdown
lag 256
exit

Do the same configuration on the VSX Peer switch (Second Switch)

Connect the cables via DAC/Optical and confirm the Port-channel health

CORE01# sh lag 256
System-ID       : b8:d4:e7:d5:36:00
System-priority : 65534

Aggregate lag256 is up
 Admin state is up
 Description : VSX-LAG
 Type                        : normal
 MAC Address                 : b8:d4:e7:d5:36:00
 Aggregated-interfaces       : 1/1/55 1/1/56
 Aggregation-key             : 256
 Aggregate mode              : active
 Hash                        : l3-src-dst
 LACP rate                   : slow
 Speed                       : 200000 Mb/s
 Mode                        : trunk


-------------------------------------------------------------------

CORE02# sh lag 256
System-ID       : b8:d4:e7:d5:f3:00
System-priority : 65534

Aggregate lag256 is up
 Admin state is up
 Description : VSX-LAG
 Type                        : normal
 MAC Address                 : b8:d4:e7:d5:f3:00
 Aggregated-interfaces       : 1/1/55 1/1/56
 Aggregation-key             : 256
 Aggregate mode              : active
 Hash                        : l3-src-dst
 LACP rate                   : slow
 Speed                       : 200000 Mb/s
 Mode                        : trunk


Form the VSX Cluster

under the configuration mode, go in to the VSX context by entering “vsx” and issue the following commands on both switches

CORE01#

vsx
    inter-switch-link lag 256
    role primary
    linkup-delay-timer 30

-------------------------------

CORE02#

vsx
    inter-switch-link lag 256
    role secondary
    linkup-delay-timer 30

Check the VSX Status

CORE01# sh vsx status
VSX Operational State
---------------------
  ISL channel             : In-Sync
  ISL mgmt channel        : operational
  Config Sync Status      : In-Sync
  NAE                     : peer_reachable
  HTTPS Server            : peer_reachable

Attribute           Local               Peer
------------        --------            --------
ISL link            lag256              lag256
ISL version         2                   2
System MAC          b8:d4:e7:d5:36:00   b8:d4:e7:d5:f3:00
Platform            8325                8325
Software Version    GL.10.06.0001       GL.10.06.0001
Device Role         primary             secondary

----------------------------------------

CORE02# sh vsx status
VSX Operational State
---------------------
  ISL channel             : In-Sync
  ISL mgmt channel        : operational
  Config Sync Status      : In-Sync
  NAE                     : peer_reachable
  HTTPS Server            : peer_reachable

Attribute           Local               Peer
------------        --------            --------
ISL link            lag256              lag256
ISL version         2                   2
System MAC          b8:d4:e7:d5:f3:00   b8:d4:e7:d5:36:00
Platform            8325                8325
Software Version    GL.10.06.0001       GL.10.06.0001
Device Role         secondary           primary

Setup the Keepalive Link

its recommended to set up a Keepalive link to avoid Split-brain scenarios if the ISL goes down, We are trying to prevent both switches from thinking they are the active devices creating STP loops and other issues on the network

This is not a must-have, it’s nice to have, As of Aruba CX OS 10.06.x you need to sacrifice one of your data ports for this

Dell OS10 VLT archives this via the OOBM network ports, Supposedly Keepalive over OOBM is something Aruba is working on for future releases

Few things to note

  • It’s recommended using a routed port in a separate VRF for the keepalive link
  • can use a 1Gbps link for this if needed

Provision the port and VRF

CORE01#

vrf KEEPALIVE

interface 1/1/48
no shutdown
vrf attach KEEPALIVE
description VSX-keepalive-Link
ip address 192.168.168.1/24
exit

-----------------------------------------

CORE02#

vrf KEEPALIVE

interface 1/1/48
no shutdown
vrf attach KEEPALIVE
description VSX-keepalive-Link
ip address 192.168.168.2/24
exit


Define the Keepalive link

Note – Remember to define the vrf id in the keepalive statement

Thanks /u/illumynite for pointing that out

CORE01#

vsx
    inter-switch-link lag 256
    role primary
    keepalive peer 192.168.168.2 source 192.168.168.1 vrf KEEPALIVE
    linkup-delay-timer 30

-----------------------------------------

CORE02#

vsx
    inter-switch-link lag 256
    role secondary
    keepalive peer 192.168.168.1 source 192.168.168.2 vrf KEEPALIVE
    linkup-delay-timer 30

Next up…….

  • VSX MC-LAG
  • VSX Active forwarding
  • VSX Active gateway

References

AOS-CX 10.06 Virtual SwitchingExtension (VSX) Guide

As always if you notice any mistakes please do let me know in the comments

IP version 6 with Dual-stack using a Tunnel broker 6in4 – PFSense/ASA -Part 01

If your ISP doesn’t have Native IP version 6 Support with Dual Stack  here is a workaround to get it setup for your home lab enviroment

What you need

> Router/Firewall/UTM that supports IPv6 Tunneling

  • PFsense/OpenSense/VyOS
  • DD-WRT 
  • Cisco ISR
  • Juniper SRX

> Active Account with an Ipv6 Tunnel Broker

      For this example we are going to be using Hurricane Electric Free IPv6 Tunnel Broker

Overview of the setup

For part 1 of this series  we are going to cover the following

  • Dual Stack Setup
  • DHCPV6 configuration and explanation

– Guide –

I used my a Netgate router running PfSense to terminate the 6in4 tunnel.it adds the firewall and monitoring capabilities on your Ipv6 network

Before we begin, we need to make a few adjustments on the firewall

Allow IPv6 Traffic

On new installations of pfSense after 2.1, IPv6 traffic is allowed by default. If the configuration on the firewall has been upgraded from older versions, then IPv6 would still be blocked. To enable IPv6 traffic on PFsense, perform the following:

  • Navigate to System > Advanced on the Networking tab
  • Check Allow IPv6 if not already checked
  • Click Save

Allow ICMP

ICMP echo requests must be allowed on the WAN address that is terminating the tunnel to ensure that it is online and reachable.

Firewall> Rules > WAN
Create a regular tunnel.

Enter your IPv4 address as the tunnel’s endpoint address.

Note – After entering your IPv4 address, the website will check to make sure that it can ping your machine. If it cannot ping your machine, you will get an error like the one below:

You can access the tunnel information from the accounts page

While you are here go to “Advance Tab” and setup an “Update key”. (We need it later)

Create and Assign the GIF Interface

Next, create the interface for the GIF tunnel in pfSense. Complete the fields with the corresponding information from the tunnel broker configuration summary.

  • Navigate to Interfaces > (assign) on the GIF tab.
  • Click fa-plus Add to add a new entry.
  • Set the Parent Interface to the WAN where the tunnel terminates. This would be the WAN which has the Client IPv4 Address on the tunnel broker.
  • Set the GIF Remote Address in pfSense to the Server IPv4 Address on the summary.
  • Set the GIF Tunnel Local Address in pfSense to the Client IPv6 Address on the summary.
  • Set the GIF Tunnel Remote Address in pfSense to the Server IPv6 Address on the summary, along the with prefix length (typically / 64).
  • Leave remaining options blank or unchecked.
  • Enter a Description.
  • Click Save.

Example GIF Tunnel.

Assign GIF Interface

Click fa-plus on Interfaces > (Assignments)

choose the GIF interface to be used for an OPT interface. In this example, the OPT interface has been renamed WAN_HP_NET_IPv6. Click Save and Apply Changes if they appear.

 

Configure OPT Interface

With the OPT interface assigned, Click on the OPT interface from the Interfaces menu to enable it  Keep IPv6 Configuration Type set to None.

Setup the IPv6 Gateway

When the interface is configured as listed above, a dynamic IPv6 gateway is added automatically, but it is not yet marked as default.

  • Navigate to System > Routing
  • Edit the dynamic IPv6 gateway with the same name as the IPv6 WAN created above.
  • Check Default Gateway.
  • Click Save.
  • Click Apply Changes.
 
Status > Gateways to view the gateway status. The gateway will show as “Online” if the configuration is successful

Set Up the LAN Interface for IPv6

The LAN interface may be configured for static IPv6 network. The network used for IPv6 addressing on the LAN Interface is an address in the Routed /64 or /48 subnet assigned by the tunnel broker.

  • The Routed /64 or /48 is the basis for the IPv6 Address field

For this exercise we are going to use ::1 for the LAN interface IP from the Prefixes provided above

Routed /64 : 2001:470:1f07:79a::/64

Interface IP – 2001:470:1f07:79a::1

Set Up DHCPv6 and RA (Router Advertisements)

Now that we have the tunnel up and running we need to make sure devices behind the lan interface can get a IPv6 address

There are couple of ways to handle the addressing

Sateless Auto Address Configuration (SLAAC)

SLAAC just means Stateless Auto Address Configuration, but it shouldn’t be confused with Stateless DHCPv6. In fact, we are talking about two different approaches.

SLAAC is the simplest way to give an IPv6 address to a client, because it exclusively rely on Neighbor Discovery Protocol. This protocol, that we simply call NDP, allows devices on a network to discover their Layer 3 neighbors. We use it to retrieve the layer 2 reachability information, like ARP, and to find out routers on the network.

When a device comes online, it sends a Router Solicitation message. It’s basically asking “Are there some routers out there?”. If we have a router on the same network, that router will reply with a Router Advertisement (RA) message. Using this message, the router will tell the client some information about the network:

  • Who is the default gateway (the link-local address of the router itself)
  • What is the global unicast prefix (for example, 2001:DB8:ACAD:10::/64)

With these information, the client is going to create a new global unicast address using the EUI-64 technique. Now the client has an IP address from the global unicast prefix range of the router, and that address is valid over the Internet.

This method is extremely simple, and requires virtually no configuration. However, we can’t centralize it and we cannot specify further information, such as DNS settings. To do that, we need to use a DHCPv6 technique

Just like IP v4 we need to setup DHCP for the IPv6 range for the devices behind the firewall to use SLAAT

Stateless DHCPv6

Stateless DHCPv6 brings to the picture the DHCPv6 protocol. With this approach, we still use SLAAC to obtain reachability information, and we use DHCPv6 for extra items.

The client always starts with a Router Solicitation, and the router on the segment responds with a Router Advertisement. This time, the Router Advertisement has a flag called other-config set to 1. Once the client receives the message, it will still use SLAAC to craft its own IPv6 address. However, the flag tells the client to do something more.

After the SLAAC process succeed, the client will craft a DHCPv6 request and send it through the network. A DHCPv6 server will eventually reply with all the extra information we needed, such as DNS server or domain name.

This approach is called stateless since the DHCPv6 server does not manage any lease for the clients. Instead, it just gives extra information as needed.

Configuring IPv6 Router Advertisements

Router Advertisements (RA) tell an IPv6 network not only which routers are available to reach other networks, but also tell clients how to obtain an IPv6 address. These options are configured per-interface and work similar to and/or in conjunction with DHCPv6.

DHCPv6 is not able to send clients a router for use as a gateway as is traditionally done with IPv4 DHCP. The task of announcing gateways falls to RA.

Operating Mode: Controls how clients behave. All modes advertise this firewall as a router for IPv6. The following modes are available:

  • Router Only: Clients will need to set addresses statically
  • Unmanaged: Client addresses obtained only via Stateless Address Autoconfiguration (SLAAC).
  • Managed: Client addresses assigned only via DHCPv6.
  • Assisted: Client addresses assigned by either DHCPv6 or SLAAC (or both).

Enable DHCPv6 Server on the interface

Setup IPv6 DNS Addresses

we are going to use cloud-flare DNS (At the time of writing CF is rated as the fastest resolver by Thousandeyes.com)

https://developers.cloudflare.com/1.1.1.1/setting-up-1.1.1.1/

1.1.1.1

  • 2606:4700:4700::1111
  • 2606:4700:4700::1001

Keeping your Tunnel endpoint Address Updated with your Dynamic IP

This only applies if you have a dynamic IPv4 from your ISP

As you may remember from our first step when registering the 6in4 tunnel on the website we had to enter our Public IP and enable ICMP

We need to make sure we keep this updated when our IP changes ovetime

There are few ways to accomplish this

  • Use PFsense DynDNS feature 

dnsomatic.com  is wonderful free service to update your dynamic IP on multiple locations, i used this because if needed i have the freedom to change routers/firewalls with out messing up my config (Im using a one of my RasPi’s to update DNS-O-Matic)

im working on another article for this, will link it to this section ASAP

 

Few Notes –

Android OS, Chrome OS still doesn’t support DHCPv6

Mac OSX and windows 10, Server 2016 uses and prefers Ipv6

Check the windows firewall rules if you have issues with NAT rules and manually update rules

Your MTU will drop-down since you are sending the IPv6 headers encapsulated in the Ipv4 packets.Personally i have no issues with my Ipv6 network Behind a spectrum DOCSIS modem. but this may cause issues depending on your ISP ie : CGNat

Here is a good write up https://jamesdobson.name/post/mtu/

 

Part 2

With Part two of this series we will use an ASA for IPv6 using the PFsense router as an tunnel-endpoint

Example Network

Link spotlight

– Understanding IPv6 EUI-64 Bit Address

– IPv6 Stateless Auto Configuration

– Configure the ASA to Pass IPv6 Traffic

– Setup IPv6 TunnelBroker – NetGate

– ipv6-at-home Part 1 | Part II | Part III

Until next time….