Connecting the world…

Getting your AOS-CX switch in Central

Everybody is talking about Cloud Management and since Aruba Central is upgraded to 2.5.2, there is the ability to manage your AOS-CX switch in Central via Template Groups.

To get this done, it is necessary to get your switch connected to Central and this isn’t always a matter of booting the switch, configure IP address, routing, DNS server and you are ready to go.

After setting up the default settings (and yes the switch has internet access) I see the following message on the switch.

6200F(config)# show aruba-central
Central admin state : enabled

Central location :
VRF for connection : default
Central connection status : connection_failure

Central source : dhcp
Central source connection status : connection_failure
Central source last connected on : N/A
System time synchronized from Activate : False

Activate Server URL :
CLI location : N/A

The switch doesn’t show up in Central. Some debugging on the switch (“debug central all” with “show debug buffer reverse”) and checking the logging on the switch tells us the following:

2020-11-03T12:17:29.456931+01:00 6200F hpe-restd[1005]: Event|7709|LOG_WARN|UKWN|1|Certificate * rejected due to verification failure (30)

So I decided to import the certificate chain from *, just like you do for downloadable user-roles in regards to ClearPass. Aruba Central uses the following certificate chain.

The COMODO RSA Certificate Authority certificate can be downloaded via the following link:

The link shows the certificate in PEM format, which can be used to add the PKI TA-Profile to the switch.

6200F(config)# crypto pki ta-profile COMODO_RSA_CA
6200F(config-ta-COMODO_CA)# ta-certificate import terminal
Paste the certificate in PEM format below, then hit enter and ctrl-D:
6200F(config-ta-cert)# —–BEGIN CERTIFICATE—–
6200F(config-ta-cert)# MIIF2DCCA8CgAwIBAgIQTKr5yttjb+Af907YWwOGnTANBgkqhkiG9w0BAQwFADCB
6200F(config-ta-cert)# hTELMAkGA1UEBhMCR0IxGzAZBgNVBAgTEkdyZWF0ZXIgTWFuY2hlc3RlcjEQMA4G
6200F(config-ta-cert)# A1UEBxMHU2FsZm9yZDEaMBgGA1UEChMRQ09NT0RPIENBIExpbWl0ZWQxKzApBgNV
6200F(config-ta-cert)# BAMTIkNPTU9ETyBSU0EgQ2VydGlmaWNhdGlvbiBBdXRob3JpdHkwHhcNMTAwMTE5
6200F(config-ta-cert)# EkdyZWF0ZXIgTWFuY2hlc3RlcjEQMA4GA1UEBxMHU2FsZm9yZDEaMBgGA1UEChMR
6200F(config-ta-cert)# Q09NT0RPIENBIExpbWl0ZWQxKzApBgNVBAMTIkNPTU9ETyBSU0EgQ2VydGlmaWNh
6200F(config-ta-cert)# dGlvbiBBdXRob3JpdHkwggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoICAQCR
6200F(config-ta-cert)# 6FSS0gpWsawNJN3Fz0RndJkrN6N9I3AAcbxT38T6KhKPS38QVr2fcHK3YX/JSw8X
6200F(config-ta-cert)# pz3jsARh7v8Rl8f0hj4K+j5c+ZPmNHrZFGvnnLOFoIJ6dq9xkNfs/Q36nGz637CC
6200F(config-ta-cert)# 9BR++b7Epi9Pf5l/tfxnQ3K9DADWietrLNPtj5gcFKt+5eNu/Nio5JIk2kNrYrhV
6200F(config-ta-cert)# /erBvGy2i/MOjZrkm2xpmfh4SDBF1a3hDTxFYPwyllEnvGfDyi62a+pGx8cgoLEf
6200F(config-ta-cert)# Zd5ICLqkTqnyg0Y3hOvozIFIQ2dOciqbXL1MGyiKXCJ7tKuY2e7gUYPDCUZObT6Z
6200F(config-ta-cert)# +pUX2nwzV0E8jVHtC7ZcryxjGt9XyD+86V3Em69FmeKjWiS0uqlWPc9vqv9JWL7w
6200F(config-ta-cert)# qP/0uK3pN/u6uPQLOvnoQ0IeidiEyxPx2bvhiWC4jChWrBQdnArncevPDt09qZah
6200F(config-ta-cert)# SL0896+1DSJMwBGB7FY79tOi4lu3sgQiUpWAk2nojkxl8ZEDLXB0AuqLZxUpaVIC
6200F(config-ta-cert)# u9ffUGpVRr+goyhhf3DQw6KqLCGqR84onAZFdr+CGCe01a60y1Dma/RMhnEw6abf
6200F(config-ta-cert)# Fobg2P9A3fvQQoh/ozM6LlweQRGBY84YcWsr7KaKtzFcOmpH4MN5WdYgGq/yapiq
6200F(config-ta-cert)# crxXStJLnbsQ/LBMQeXtHT1eKJ2czL+zUdqnR+WEUwIDAQABo0IwQDAdBgNVHQ4E
6200F(config-ta-cert)# FgQUu69+Aj36pvE8hI6t7jiY7NkyMtQwDgYDVR0PAQH/BAQDAgEGMA8GA1UdEwEB
6200F(config-ta-cert)# /wQFMAMBAf8wDQYJKoZIhvcNAQEMBQADggIBAArx1UaEt65Ru2yyTUEUAJNMnMvl
6200F(config-ta-cert)# wFTPoCWOAvn9sKIN9SCYPBMtrFaisNZ+EZLpLrqeLppysb0ZRGxhNaKatBYSaVqM
6200F(config-ta-cert)# 4dc+pBroLwP0rmEdEBsqpIt6xf4FpuHA1sj+nq6PK7o9mfjYcwlYRm6mnPTXJ9OV
6200F(config-ta-cert)# 2jeDchzTc+CiR5kDOF3VSXkAKRzH7JsgHAckaVd4sjn8OoSgtZx8jb8uk2Intzna
6200F(config-ta-cert)# FxiuvTwJaP+EmzzV1gsD41eeFPfR60/IvYcjt7ZJQ3mFXLrrkguhxuhoqEwWsRqZ
6200F(config-ta-cert)# CuhTLJK7oQkYdQxlqHvLI7cawiiFwxv/0Cti76R7CZGYZ4wUAc1oBmpjIXUDgIiK
6200F(config-ta-cert)# boHGhfKppC3n9KUkEEeDys30jXlYsQab5xoq2Z0B15R97QNKyvDb6KkBPvVWmcke
6200F(config-ta-cert)# jkk9u+UJueBPSZI9FoJAzMxZxuY67RIuaTxslbH9qh17f4a+Hg4yRvv7E491f0yL
6200F(config-ta-cert)# S0Zj/gA0QHDBw7mh3aZw4gSzQbzpgJHqZJx64SIDqZxubw5lT2yHh17zbqD5daWb
6200F(config-ta-cert)# QOhTsiedSrnAdyGN/4fy3ryM7xfft0kL0fJuMAsaDk527RH89elWsn2/x20Kk4yl
6200F(config-ta-cert)# 0MC2Hb46TpSi125sC8KKfPog88Tk5c0NqMuRkrF8hey1FGlmDoLnzc7ILaZRfyHB
6200F(config-ta-cert)# NVOFBkpdn627G190
6200F(config-ta-cert)# —–END CERTIFICATE—–
The certificate you are importing has the following attributes:
Subject: C = GB, ST = Greater Manchester, L = Salford, O = COMODO CA Limited, CN = COMODO RSA Certification Authority
Issuer: C = GB, ST = Greater Manchester, L = Salford, O = COMODO CA Limited, CN = COMODO RSA Certification Authority
Serial Number: 0x4CAAF9CADB636FE01FF74ED85B03869D
TA certificate import is allowed only once for a TA profile
Do you want to accept this certificate (y/n)? y

The connection to Central is successful after adding the PKI TA-Profile for the COMODO certificate. You are now ready to get going with the template configuration for the AOS-CX switch in Aruba Central.


Of course you can always try to debug the Central connection with the commando’s

debug central all
debug destination buffer
show debug buffer reverse

phpIPAM – Azure and SAML authentication

What is easier than using your Azure credentials to log in to your web applications like phpIPAM? My daily job is networking, like routing, switching, wireless, and Wi-Fi, so I had to puzzle when I had to configure SAML2 authentication between phpIPAM and our company Azure infrastructure. I couldn’t find a lot of information about the configuration, but I managed to get it done eventually.

The only thing that doesn’t seem to work is the mapping between a local user and the Azure user. Every successful authentication from Azure ends up by signing in with the default phpIPAM admin user. I can live with that (for now). To get the configuration done, you need to follow these steps:

  • Access the Azure portal via
  • Navigate to Azure Active Directory
  • Choose Enterprise Applications
  • Configure a New Application
  • Choose the option Non-gallery application
  • Configure a name for the application and choose Add

The next step is to configure the single sign on part of Azure.

  • Choose option 2. Set up single sign on from the above image
  • Edit the Basic SAML Configuration
  • Enter the following information (our phpIPAM URL is
    • Identifier (Entity ID):
    • Reply URL (Assertion Consumer Service URL):
  • Save the configuration

The next step involves downloading the SAML Signing Certificate and get the certificate fingerprint. You can download the certificate under 3 SAML Signing Certificate.

Download and save the Base64 certificate. I use OpenSSL and the following command to retrieve the fingerprint:

openssl x509 -in saml-cert.cer -fingerprint -sha256 -noout -text

You find the fingerprint at the end of the output. Save the fingerprint. The last piece of information is the links to Azure AD. Mine can be found below.

You have all the information to go ahead and configure the SAML connection in phpIPAM. The last step is to add users or groups to the Enterprise application. Choose Users and Groups from the menu and add the users or groups which are allowed to have access. We don’t have a premium subscription, so I can only add single users and no groups.

This is it for the Azure part. The next step is the phpIPAM configuration. Log in to your phpIPAM environment and go to the Authentication Methods in phpIPAM and create a new SAML2 Authentication. Enter all the information from the Azure portal into the correct fields.

Use the following information.

IDP issuerAzure AD Identifier
IDP login urlLogin URL
IDP logout urlLogout URL
IDP cert fingerprintOpenssl fingerprint

Add the SAML2 Authentication Method. The next step would be to create a local user in phpIPAM and associate the user with the SAML2 Authentication Method.

You are now ready to test your SAML integration. I now have the opportunity to log in with my Azure account, including MFA, to access the phpIPAM environment.

User tunnel not operational

HPE Aruba switches have the concept of user-based tunnelling. In short, the wired connections behave like a wireless connection. All traffic from the wired client is tunnelled to the central controller. This provides functions like central firewalling and micro-segmentation by blocking inter-user traffic.

Yesterday I had a customer complaining that multiple clients weren’t able to communicate. After investigation the problem focused on one HPE 2930F stack. The stack had been working without problems, but now I found the following error message in the logging.

I 01/17/20 10:16:01 05563 dca: ST1-CMDR: Failed to apply user role
THIN_CLIENT_UBT-3007-3_7Z4q with tunnel redirect to 8021X
client F44D306E2DB9 on port 3/21 as user tunnel is not operational.

I couldn’t find a lot on the internet concerning this issue. I only found something on EventID 5563 in the Aruba Event Log Reference Message Guide for ArubaOS-Switch 16.08.

The EventID description is This log event informs the user that Tunneled-node-server-redirect is enabled in the user role but per user tunnel feature is disabled.

I checked the switch “show tunneled-node-server”, and the feature is enabled. I deleted the “tunneled-node-server” configuration and reapplied the configuration to the switch, but still the same error message.


Jan 17 07:39:10  stm[4064]: <304109> <5640> <WARN> |stm|  No available license type PEFNG for Tunneled Node 54:80:28:cf:4a:4b

A switch consumes a license for user-based tunnelling.

AOS – WireShark: remote capture

AOS switches have the option to monitor / copy traffic from port A to port B. You also have the option to send the monitor traffic to a remote switch or even to a remote host. When the remote host is running WireShark, the monitored traffic can be analysed on the remote host.

First you need to configure the switch to send a copy of the traffic to a remote host. Use the following commands to create a monitor session to a remote host. In this case the switch is using IP adres with source port UDP/10999 and the remote host has IP adres

ASW-C01# conf t
ASW-C01(config)# monitor 
 mac                   MAC address.
ASW-C01(config)# mirror 
 endpoint              Remote mirroring destination configuration.
 <1-4>                 Mirror destination number.
ASW-C01(config)# mirror 1 
 name                  Mirroring destination name string.
 port                  Mirroring destination monitoring port.
 remote                Remote mirroring destination configuration.
ASW-C01(config)# mirror 1 remote 
 ip                    Remote mirroring destination configuration.
ASW-C01(config)# mirror 1 remote ip 
 IP-ADDR               Enter an IP address.
ASW-C01(config)# mirror 1 remote ip 
 <1-65535>             Remote mirroring UDP encapsulation port.
ASW-C01(config)# mirror 1 remote ip 10999 
 IP-ADDR               Remote mirroring UDP encapsulation destination ip addr.
ASW-C01(config)# mirror 1 remote ip 10999 
 truncation            Enable truncation for Remote mirroring.
ASW-C01(config)# mirror 1 remote ip 10999 
The destination switch must be configured before proceeding.

Has the remote switch been configured (y/n)? y

Next you need to configure the interface for which you would like to analyse the traffic.

ASW-C01(config)# int 4/3
ASW-C01(eth-4/3)# monitor 
 all                   Monitor all traffic.
ASW-C01(eth-4/3)# monitor all both 
 mirror                Mirror destination.
ASW-C01(eth-4/3)# monitor all both mirror 1 
 no-tag-added          Don’t add VLAN tag for this untagged-port
 <1-4>                 Mirror destination number.
ASW-C01(eth-4/3)# monitor all both mirror 1 

Traffic from port 4/3 is now send to the remote host. Now start WireShark on the remote host and create a capture filter to capture only packets for port UDP/10999.

WireShark displays packets like below, which are useless to analyse traffic. The packets are encoded as HP ERM packets.

So the final step is to decode the traffic. Just right click on a packet and choose the option “Decode As…”. You could also choose from the menu Analyze >> Decode As…

Change the column Current from (none) to HP_ERM from the drop down list and choose OK.

HP ERM, Hewlett-Packard Encapsulated Remote Mirror protocol is used by the HPE (Hewlett-Packard Enterprise) switches based on ProVision ASICs formerly of the ProCurve family, now branded under Aruba Networks, a Hewlett Packard Enterprise company. Unlike Cisco RSPAN, HP ERM encapsulates the frames to be mirrored inside UDP datagrams with a proprietary header, allowing it to be transported over any IP network (like Cisco ERSPAN)

Now the packets should be “readable” for traffic analysis.

Migrate RAP from AOS 6.x to AOS 8.x

I guess something that many HPE Aruba wireless engineers have to do these days is migrating the “old” AOS 6.x environment to the new AOS 8.x with Mobility Masters. I am not going to explain what the differences between both are and what a Mobility Master does, but I have a tip when you need to migrate remote access points (RAPs).

Yesterday evening I had to perform a migration from 3 7030 controllers to a environment with redundant Mobility Masters. The solution contains 35 campus APs and 20 remote APs. The migration should be relatively simple…….

I installed the Mobility Masters, added the licences, create the node hierarchy, add all the configuration regarding WiFi, AAA, RAP ports, user-roles, VLANs and so on. To make live easier I exported the whitelist db on the old controller (localuserdb export) and imported the whitelist db into the AOS 8.4 (or 8.5) environment (localuserdb import).

I was able to add one controller to the Mobility Masters by distributing the APs in the 6.5 environment over the remaining 2 controllers. So the old environment had 2 controllers left, one for the campus APs and one for the RAPs. I wanted to migrate the campus APs first. Can’t be that difficult……. just provision the campus AP with the IP address of the AOS 8.4 controller and wait for the magic to happen. All campus AP came online, so I added the second controller to the Mobility Master.

Since I now have two controllers I started with the lc-cluster configuration. I configured the lc-cluster group-profile, added the two controllers, including the vrrp-ip and rap-public-ip, like shown below.

lc-cluster group-profile “lc-booches”
controller priority 128 mcast-vlan 0 vrrp-ip vrrp-vlan 1 group 0 rap-public-ip
controller priority 128 mcast-vlan 0 vrrp-ip vrrp-vlan 1 group 0 rap-public-ip

Only two more steps left to add the RAPs:

  1. Add the correct NAT and firewall configuration to the firewall environment
  2. Add a LC-RAP-pool to the Mobility Master >> /mm level

BAM, ready to migrate the RAP. I converted the first by reprovision the AP in the old controller and configure the new public IP. Here we go!! You only need to wait and wait and wait and wait……. but the RAP didn’t come online!!! What can go wrong??

  • Created a whitelist entry? – yes, export / import and I see the entries via show whitelist-db rap
  • Created a LC-RAP-pool? – yes, show lc-rap-pool on /mm level
  • Correct LC-cluster config? – yes, rap-public-ip is configured correctly
  • Firewall config? – Config is correct. Firewall policy is matched and packet sniffer shows that packets are forwarded to controller
  • Controller hardware limit reached? No, there are currently 35 campus APs and the limit of a 7030 is 64 APs

Time for some deeper troubleshooting. The RAPs gets stuck at the logon role in the user-table. Time for some crypto debugging. Lets get started with logging security process crypto level debugging to check if a VPN tunnel establishment.

Hhhhmmmm, there is a strange error message!!!

|ike| HMAC_SHA1_96 ESN_0 <– R Notify: INTERNAL_ADDRESS_FAILURE (ESP spi=ea87a200)#SEND 80

Seems like the RAP IP pool isn’t configured correctly. So double check, but the pool is definitely configured correctly. Okay, another test. One RAP is onsite, so factory default the RAP and convert it from scratch. SAME RESULT AND SAME ERROR message. Hhhhmmm, I am AMFX certificated, but can’t even get a RAP connected?!?!?! So it is time to ask another AMFX’er, my colleague Peter (AMFX#36). Luckily he was willing to power up his RAP and provision it to the customer’s environment.

Okay, let’s add the RAP to the whitelist, let Peter do the conversion magic from IAP to RAP and check what happens. BAM, RAP connects, updates software and is online……. Difference, other hardware. Our RAP is RAP303-H, the customer is using RAP155P. RAP155P is supported in AOS 8.4, so that can’t be the problem.

So what is another difference…. THE WHITELIST. Our RAP was added manually after the configuration was completely done and wasn’t imported at the beginning. So let’s check the whitelist again and there it is!!!! I didn’t really check all columns from the show whitelist-db rap, but one of the columns on the right is called Cluster-InnerIP. The output below shows the column, normally there are even more columns, so you have to scroll to view the Cluster-InnerIP.

You notice that the manually added entry gets an IP address assigned from the LC-RAP-pool and all imported RAPs don’t get an IP address. This definitely reflexes the error INTERNAL_ADDRESS_FAILURE error message. The cert-type is changed to factory-cert after the RAP is converted and online.

So I removed one of the imported entries and added it again (manually) to the whitelist db and guess what. The Cluster-InnerIP is assigned to the RAP and the RAP connects directly and without any problem.

So in my opinion there could be two reasons why this issue arrises:

  1. I imported the whitelist-db to early, because I hadn’t configured the LC-RAP-pool yet;
  2. The import from 6.5 to 8.4 doesn’t work for RAP entries;

To be sure, I removed all entries from the whitelist db and imported the whitelist again. Again the same result, the RAPs don’t get a Cluster-InnerIP assigned. Conclusion:

TIP of the WEEK: when migrating from 6.5 to 8.4 MANUALLY add the whitelist-db rap entries to the 8.4 environment!!!

Or does someone have a better way to import the whitelist for remote APs?!?!?!

1 2 3 54