Cisco WLC – HA SSO upgrade

“Is the upgrade procedure for a high-availability pair of Cisco Wireless LAN Controllers the same as the procedure for a single Cisco WLC?” Several customers asked me this questions and the answer is YES.

First you check the current and backup firmware image.

(Cisco Controller) >show boot
Primary Boot Image…………………………. 8.2.111.0 (default) (active)
Backup Boot Image………………………….. 8.1.102.0

Next you check if your SSO configuration is working as expected.

(Cisco Controller) >show redundancy summary
Redundancy Mode = SSO ENABLED
Local State = ACTIVE
Peer State = STANDBY HOT
Unit = Primary
Unit ID = 00:81:C4:87:3B:C9
Redundancy State = SSO
Mobility MAC = 00:81:C4:87:3B:C9
BulkSync Status = Complete
Average Redundancy Peer Reachability Latency = 177 Micro Seconds
Average Management Gateway Reachability Latency = 935 Micro Seconds

Upload the new firmware to the controller by using an TFTP or FTP server. I am using an TFTP server in this example.

(Cisco Controller) >transfer download datatype code
(Cisco Controller) >transfer download filename AIR-CT5520-K9-8-2-141-0.aes
(Cisco Controller) >transfer download path .
(Cisco Controller) >transfer download serverip 10.200.8.83
(Cisco Controller) >transfer download mode tftp
(Cisco Controller) >transfer download start

After the TFTP session is finished you’ll notice that the the software is automatically transferred from the active to the standby unit.

TFTP Code transfer starting.

TFTP receive complete… extracting components.

Checking Version Built.

Image version check passed.

Informing the standby to start the transfer download process

Waiting for the Transfer & Validation result from Standby.

Standby – Standby receive complete… extracting components.

Standby – Image version check passed.

Transfer & validation on Standby success, proceed to Flash write on Active.

Writing new AP Image Bundle to flash disk.

Executing fini script.

File transfer is successful.
Reboot the controller for update to complete.
Optionally, pre-download the image to APs before rebooting to reduce network downtime.

Transfer Download complete on Active & Standby

The last step is to reload both controllers to activate the firmware. After you reboot the active controller, you are able to access the standby controller and reboot that controller too. You have the option to reboot both controllers with one command.

(Cisco Controller) >reset system both in 00:05:00 image no-swap reset-aps

The controller also has the option to pre download the firmware from the controller to the access-points. This speeds up the upgrade process for the access-points, because the access-point don’t need to download the firmware after the controllers are online again. The access-point only need to reboot when the loose the connection with the controller. I will describe this process in a separate post.

After the controllers are back online, you should check the primary and backup boot firmware to see if the upgrade was successful.

(Cisco Controller) >show boot
Primary Boot Image…………………………. 8.2.141.0 (default)
Backup Boot Image………………………….. 8.2.111.0 (active)

Cisco FMC – Dashboard Widgets

Some widgets on the dashboard don’t generate graphs after deploying a default configuration of Cisco FireSight Management Center.

The first two widgets, Top Server Applications Seen and Top Operating Systems Seen, are generated after the configuration of a Network Discovery Profile. The configuration of the Network Discover Profile is done via Policies – Network Discovery – Networks. I always configure a Network Discovery Profile to profile all Hosts, Users and Application for the RFC1918 IP address space.

To generate graphs for the URL widgets, you need to make sure that the correct options for the URL filtering service are enable. The URL filtering service configuration is done via System – Integration – Cisco CSI. I use the following settings for URL filtering.

After this you should wait a while (about one hour) to check if the graphs are generated. If you don’t want to wait, you can check the Analysis tab to see if information is gathered and displayed by the Cisco FireSight Management Center appliance.

Cisco ASA: multiple context and capture

Packet captures are very useful for troubleshooting purposes. The Cisco ASA supports packet captures even in multiple context mode. I normally configure packet captures on CLI level. This can be done by configuring an access-list to match the specific traffic you would like to capture. Add the access-list and the specific interface in a capture command. Mostly I download the capture in raw format for further analysis with a tool like WireShark. The capture can be downloaded via TFTP or via a secure connection (HTTPS) to the Cisco ASA firewall.

When running a Cisco ASA in multiple context mode, I always disable the ability to connect directly to a context for management purposes. That way you have to access the admin context for management access, but this also denies the option to download the capture via a secure connection directly from the Cisco ASA traffic context.

The easiest way to download the capture in multiple context mode is via a TFTP transfer from the system context. Check the example command below. The capture is made within the context named contextA and the capture has the name captureA. The following command can be used to download the capture in raw (pcap) format.

copy /pcap capture:contextA/captureA tftp://10.10.10.10/captureA.pcap

You can now analyse the capture with WireShark

Cisco IOS-XE 16.x

Cisco has release new IOS-XE software, called IOS-XE Denali 16.x. This software is available for Cisco ASR routers and Cisco Catalyst 3850/3650 switches. In the end IOS-XE Denali should be available for all switches.

A good overview of Cisco Catalyst IOS XE Denali is explained in this Youtube video from Tech Field Day.

Below you see the commands to upgrade a Cisco Catalyst 3850 switch stack consisting 3 switches to the new IOS-XE Denali firmware. When you use the default software install commands you will receive an error message, like shown below.

C3850#software install file flash:/cat3k_caa-universalk9.16.01.02.SPA.bin switch 1-3 on-reboot
Preparing install operation …
[2]: Copying software from active switch 2 to switches 1,3
[2]: Finished copying software to switches 1,3
[1 2 3]: Starting install operation
[1 2 3]: Expanding bundle flash:cat3k_caa-universalk9.16.01.02.SPA.bin
[1 2 3]: Copying package files
[1 2 3]: Package files copied
[1 2 3]: Finished expanding bundle flash:cat3k_caa-universalk9.16.01.02.SPA.bin
[1 2 3]: Verifying and copying expanded package files to flash:
[1 2 3]: Verified and copied expanded package files to flash:
[1 2 3]: Starting compatibility checks
[1]: % Candidate package compatibility checks failed because the following
package dependencies were not satisfied. Operation aborted.

[2]: % Candidate package compatibility checks failed because the following
package dependencies were not satisfied. Operation aborted.

[3]: % Candidate package compatibility checks failed because the following
package dependencies were not satisfied. Operation aborted.

To get a successful install, you need to add the commands new and force, like shown in the output below. You need to manually reboot the switch during a maintenance windows to active the Cisco IOS-XE Denali firmware.

C3850#software install file flash:/cat3k_caa-universalk9.16.01.02.SPA.bin switch 1-3 new force on-reboot
Preparing install operation …
[2]: Copying software from active switch 2 to switches 1,3
[2]: Finished copying software to switches 1,3
[1 2 3]: Starting install operation
[1 2 3]: Expanding bundle flash:cat3k_caa-universalk9.16.01.02.SPA.bin
[1 2 3]: Copying package files
[1 2 3]: Package files copied
[1 2 3]: Finished expanding bundle flash:cat3k_caa-universalk9.16.01.02.SPA.bin
[1 2 3]: Verifying and copying expanded package files to flash:
[1 2 3]: Verified and copied expanded package files to flash:
[1 2 3]: Starting compatibility checks
[1 2 3]: Bypassing peer package compatibility checks due to ‘force’ command option
[1 2 3]: Finished compatibility checks
[1 2 3]: Starting application pre-installation processing
[1 2 3]: Finished application pre-installation processing
[1]: Old files list:
Removed cat3k_caa-base.SPA.03.07.01E.pkg
Removed cat3k_caa-drivers.SPA.03.07.01E.pkg
Removed cat3k_caa-infra.SPA.03.07.01E.pkg
Removed cat3k_caa-iosd-universalk9.SPA.152-3.E1.pkg
Removed cat3k_caa-platform.SPA.03.07.01E.pkg
Removed cat3k_caa-wcm.SPA.10.3.110.0.pkg
[2]: Old files list:
Removed cat3k_caa-base.SPA.03.07.01E.pkg
Removed cat3k_caa-drivers.SPA.03.07.01E.pkg
Removed cat3k_caa-infra.SPA.03.07.01E.pkg
Removed cat3k_caa-iosd-universalk9.SPA.152-3.E1.pkg
Removed cat3k_caa-platform.SPA.03.07.01E.pkg
Removed cat3k_caa-wcm.SPA.10.3.110.0.pkg
[3]: Old files list:
Removed cat3k_caa-base.SPA.03.07.01E.pkg
Removed cat3k_caa-drivers.SPA.03.07.01E.pkg
Removed cat3k_caa-infra.SPA.03.07.01E.pkg
Removed cat3k_caa-iosd-universalk9.SPA.152-3.E1.pkg
Removed cat3k_caa-platform.SPA.03.07.01E.pkg
Removed cat3k_caa-wcm.SPA.10.3.110.0.pkg
[1]: New files list:
Added cat3k_caa-rpbase.16.01.02.SPA.pkg
Added cat3k_caa-rpcore.16.01.02.SPA.pkg
Added cat3k_caa-srdriver.16.01.02.SPA.pkg
Added cat3k_caa-wcm.16.01.02.SPA.pkg
Added cat3k_caa-webui.16.01.02.SPA.pkg
[2]: New files list:
Added cat3k_caa-rpbase.16.01.02.SPA.pkg
Added cat3k_caa-rpcore.16.01.02.SPA.pkg
Added cat3k_caa-srdriver.16.01.02.SPA.pkg
Added cat3k_caa-wcm.16.01.02.SPA.pkg
Added cat3k_caa-webui.16.01.02.SPA.pkg
[3]: New files list:
Added cat3k_caa-rpbase.16.01.02.SPA.pkg
Added cat3k_caa-rpcore.16.01.02.SPA.pkg
Added cat3k_caa-srdriver.16.01.02.SPA.pkg
Added cat3k_caa-wcm.16.01.02.SPA.pkg
Added cat3k_caa-webui.16.01.02.SPA.pkg
[1 2 3]: Creating pending provisioning file
[1 2 3]: Finished installing software. New software will load on reboot.

Cisco WLC and pre-download software to AP

A simple post, because I always forget the CLI commands to TFTP the software to the controller. I also added the command to predownload the new firmware to all access-points. This dramatically speeds up the upgrade process of the access-points.

You need to set the TFTP parameters first.

(Cisco Controller) >transfer download datatype code
(Cisco Controller) >transfer download mode tftp
(Cisco Controller) >transfer download serverip 10.77.244.196
(Cisco Controller) >transfer download path .
(Cisco Controller) >transfer download filename AIR-WLC4400-K9-5-2-178-0.aes

Next you can start the actual download of the firmware image.

(Cisco Controller) >transfer download start

You can now choose to reboot the controller without predownloading the firmware to the access-points. Predownloading the images is done via the command:

(Cisco Controller) >config ap image predownload primary all

You can view the progress of the predownload via:

(Cisco Controller) >show ap image all

Sometimes the predownloaded image is stored as backup image on the access-points. You can swap the image to the primary image via

(Cisco Controller) >config ap image swap all

Issue the following command to see the images on the Cisco WLC

(Cisco Controller) >show boot