What is a fabric?

I get the question a lot. What is a fabric? Most have an idea of what it is, but its a blurry idea. Fabric is a word that is being thrown around by many. It can be intimidating to ask, what exactly is a fabric. So today our goal is to provide an overview to the question. What is a fabric? I define it in the first section but get into some of the components throughout the post.

A fabric is a two layer construct. There is an underlay which is your physical make up of the network. Responsible for moving the packet. The overlay is the abstract service layer, it is responsible for things such as policies, segmentation, and mobility. Some overlay technologies that you may be aware of: CAPWAP, GRE, MPLS, DMVPN, OTV, ACI, LISP.

Overlays can come in two ways:

Layer 2 Overlays – Offers single subnet mobility within layer 2, extending L2 flooding.
Layer 3 Overlays –  Offers IP mobility, Transports IP packets, contains failure domains.

So the new term for a LAN enabled fabric is ‘campus fabric’. However, its commonly shortened and referred to as a fabric or a fabric domain.

The campus fabric can be summarized as having

  1. LISP based Control-Pane
  2. VXLAN based Data-Plane
  3. Integrated Cisco TrustSec (policy plane)

Let’s pause to define some terms within the Campus Fabric. Don’t glaze over on me! Just be aware of these terms, no need to know them inside and out for this overview.

Control-Plane Node = LISP Map-Server : is a node in the fabric responsible for resolving EIDs to RLOCs
Edge Node = LISP Tunnel Router (xTR) : responsible for encapsulating the packet at the edge of the network and authenticating endpoints.
Border Node = LISP Proxy Tunnel Router (PxTR) : responsible for de-encapsulating the packet or encapsulating the packet when entering or leaving the fabric. Any traffic entering or leaving the fabric goes through this node. You NEED at least one of these.
Intermediate Node = Non-LISP IP forwarder : Imagine a 3-tier architecture; Access-Distro-Core, the Distro would be your intermediate nodes.
Fabric Domain = FD = LISP Process
Virtual Network = VN = LISP Instance = VRF : Maintains a separate Routing and switching instance for each virtual network
Endpoint ID Group = EIG = Segment = SGT : Each user of device is assigned to a unique Endpoint ID Group
Host Pool = Dynamic EID = VLAN + IP Subnet : Provides the basic IP constructs including Anycast Gateway.

Screen Shot 2018-11-01 at 8.39.05 AM

LISP overview

Locator ID Separation Protocol – Now read that back sloooowly.. It’s self defining. LISP is separating IP(ID) from location(Locator).

LISP at it’s simplest form is nothing more than a mapping system but is a routing architecture, a control plane protocol and a data plane protocol. It separates Identity from a Location.

A devices IP (identity) is generally associated to a location of that device. If a device wants to roam to a new location the device needs a new IP (identity) for that location.

Screen Shot 2018-11-01 at 8.38.43 AM

From a design perspective you can see how this can be helpful. This is just like when you roam between cell towers with your cell phone. Your location changes but your cell phone number (identity) does not.

LISP will assign End-point Identifiers or EID to hosts and will take on the role of acting as the devices identity. I don’t want to confuse you, EIDs do not replace a devices IP. EIDs are there to help RLOC map a device to a location.

Screen Shot 2018-10-31 at 4.19.09 PM

Routing locators or RLOCs map End-point Identifiers (EID) to a current location. The control plane node is responsible for doing this. RLOCs should be viewed as the Control plane protocol for LISP.

LISP mapping system can be analogous to a DNS resolution where DNS queries answer the “WHO IS” question and LISP resolution answered the “WHERE IS” question.

Why do I care about LISP? Because the fabric overlay is LISP enabled!

VXLAN Data-Plane Overview

VXLAN Encapsulation is used to carry L2 information. This is necessary because if we were to run LISP only, we would not be able to carry L2 information. LISP payload only supports L3 information. So, to achieve L2 and L3 capabilities, we run VXLAN.

Screen Shot 2018-11-01 at 8.38.55 AM

What is Cisco TrustSec?

TrustSec (CTS) takes the complexity away from the static ACL and uses a tag based mechanism. So instead of creating enforcement on an IP you create enforcement around a tag which allows for mobility.

Lets wrap things up with a list of platforms that support campus fabric.

Fabric Edge Nodes

  • Catalyst 9300 – IOS-XE 16.6.3+
  • Catalyst 9400 – IOS-XE 16.6.3+
  • Catalyst 3K – IOS-XE 16.6.3+
  • Catalyst 4500 Sup8E+ – IOS-XE 3.9+

Fabric Border Nodes

  • Catalyst 9500 – IOS-XE 16.6.3+
  • Catalyst 3K
  • Catalyst 6800 Sup2T/6T – IOS 15.4SY+
  • ASR-1000-X/HX – IOS-XE 16.4+
  • ISR4K – IOS-XE 16.4+
  • Nexus 7700 w/ M3 cards – NX-OS 7.3.2+

Fabric Control Plane

  • Catalyst 9500
  • Catalyst 3K
  • Catalyst 6800 Sup2T/6T
  • ASR-1000-X/HX
  • ISR4K

Lab it up! You only need three switches to test this out.

You can use the command ‘fabric auto’ which will automatically generate the necessary LISP and VXLAN configurations on the device so that the device can join the fabric.

For example:

Edge(config)# fabric auto
Edge(config-fabric-auto)# domain default
Edge(config-fabric-auto)# control-plan auth-key key1
Edge(config-fabric-auto)# border

I hope that this served as a basic overview to campus fabric. It is the behind the scenes magic of intent based networking.


Converting Cisco IOS-XE Software from Bundle Mode to Install Mode

Recommended Releases for Cat9k

Today we’re are going to be converting a Cisco WS-C3850-24XS from a Bundle Running Mode to an Install Running Mode.

If you haven’t read my other post on operating modes for the Cat3k or 9Ks, look there first. Upgrading Cisco IOS-XE Software (Install Mode)

You can also review upgrade procedure for specific hardware.
Catalyst 9200 upgrade procedure or review Campus switching positioning with Catalyst 9Ks for a quick reference to determine what hardware is best suited for your campus.

I first want to show you the file(s) that each mode references. I’ll use the show version command to do this.


You can see from the previous output that the 3850 is running in BUNDLE mode. Secondly, the line that starts with ‘System image file is..” This line is the name and location of the booted Cisco IOS XE bundle file. Notice that this is a .bin extension.


Again using the show version command, in the previous output the 3650 is running in INSTALL mode. This time the line that starts with ‘System image file is..” is referencing the name and location of the provisioning file ‘packages.conf‘.

Let’s continue changing our Bundle running mode to Install running mode.

To do this, execute the command below in exec.

3850# software expand running to flash:


I am executing this on a stack so you can see that the operation is expanding the bundle (.bin) file to switch 1 and switch 2. This is essentially unpacking .pkg files from the running .bin file on the switch.

Notice that the switch attempts to create a packages.conf file but it already exists, so it creates a file called ‘running-packages.conf‘. This isn’t a big deal. If you want your file to be named packages.conf, just rename the original packages.conf to something else before you run the above command.

After this finishes, we can view the flash:/ to see our pkg files.


Here we see two .pkg versions, 03.07.04E.pkg and 03.07.05E.pkg. Which one is the most recent one? 03.07.05E.pkg is the most recent because that is the version we extracted from our current running cat3k_caa-universalk9.SPA.03.07.05.E152-3.bin file. Also, notice the running-packages.conf file.

Let’s change the boot system variable to reference our new .conf file.


Note: Check to see if you already have a boot variable defined. Change it so that on next boot you load your packages.conf file and not the .bin file. Check the boot var with the command show boot to confirm.

Save your running config to start up and reload the switch.

After the reload, we can check our running mode.


Lets clean up our flash directory.


Here is the flash directory after we cleaned it.



Upgrading Cisco IOS-XE Software (Install Mode)

Recommended Releases for Cat9k

I got my hands on a Cisco Catalyst 3650-48PD-L switch which is the first access level switch that I’ve dealt with that is running IOS-XE. IOS-XE isn’t new, I prefer it over IOS because of its Linux base and the processing advantages it has. So today we will be going through the software upgrade process for a 3650 running in install mode.

You can view my other post on how to convert IOS XE from Bundled running mode to Install running mode.

Converting Cisco IOS-XE Software from Bundle Mode to Install Mode

You can also review upgrade procedure for specific hardware.
Catalyst 9200 upgrade procedure

As a quick introduction, here are some differences between IOS and IOS-XE.


  • Monolithic – The OS and its processes run in the same address space on the same hardware.
  • A single process could crash the entire system


  • Runs a Linux OS
  • Modular System
  • Multiprocessing allows for workloads to be shared across multiple CPUs.
  • Individual sub packages create IOS-XE that can be upgraded individually

Now that you have some understanding of the differences between IOS and IOS-XE you need to understand that there is two modes of operation.

The Cisco Catalyst 3650, 3850, and 9K series switches have two modes of operation, Install Mode and Bundle Mode.

Install Mode

Install mode uses a package-provisioning file named packages.conf, which is in charge of booting the switch. There are several .pkg files found in the flash drive that provide a specific function to the OS. Cisco recommends not altering any of the files.

Bundle Mode

Bundle mode uses monolithic Cisco IOS images to boot the switch. It consumes more memory than Install mode because packages are extracted from the bundle and copied to RAM.

I would suggest operating in Install mode as this is the default, and provides you with a modular system, you can switch them if you choose.

Lets continue into our upgrade..

You can see from the output of the 3650 that the current software version is 03.06.05b.e and it is running in Install Mode.


Download your image from Cisco. The new software version here will be 03.06.06E. I have chosen to use a USB drive to place my source image on, but you can still use a transfer protocol like TFTP or SCP.


After confirming that you have your source image in its proper location, execute the following command on the switch with your source image accurately defined.

Switch#software install file usbflash0:cat3k_caa-universalk9.SPA.03.06.06.E.152-2.E6.bin

The switch prepares the image for installation; below you see the switch performing pre-installation tasks. It will ask you to confirm a reload. Confirm a reload by typing Yes.


It will take several minutes to reload, during the reload processes the switch does several post-installation tasks such as updating the Front-end Microcode.


Once the switch has successfully finished reloading, you can log back in and confirm that the new version was successfully applied. Below we have successfully loaded 03.06.06E onto the switch.


As a post clean up task you should execute the command software clean switch 1. This will remove the packages.conf and .pkg files associated to the old image.

Switch#software clean switch 1


Here is what the flash looks like post upgrade..


That wraps up the upgrade process. I hope that this has been helpful for you.



Cisco Cyber Ops Scholarship

Back in July, Cisco started a scholarship campaign for their new certification track CCNA Cyber Ops. A 10-million dollar investment would be made to increase the talent pool with critical cybersecurity proficiency.

“Through the scholarship program, Cisco will offer free training, mentoring, and testing designed to help you earn CCNA Cyber Ops certification and hone the skills needed for the job role of security operations center analyst. The new CCNA Cyber Ops certification has been designed to address the critical skills deficit, providing the job-ready knowledge needed to meet current and future challenges in network security.” -Cisco


The qualification for the scholarship were minimal, (you can find them at the above link) so I decided to apply. After months of no communications and one assessment test, I finally received this….



Congratulations to anyone that also got accepted. I’m excited to see where this takes me.