Connecting the world…

failover

Cisco CSC-SSM-20 notes

The Cisco CSC-SSM-20 modules provide advanced scanning technologies within the Cisco ASA firewall. During installations of these modules I created some quick notes, which I would like to share with you.

Initial configuration

After inserting the Cisco CSC-SSM modules into the Cisco ASA firewall, you have two ways to configure the initial configuration. The first method is through Cisco ASDM and launching the Cisco CSC-SSM Launch Wizard under Configuration. The second method is through the Cisco ASA CLI. Below you see an example of the CLI method.

session 1 do setup ac-key base 1 <license key>
session 1 do setup ac-key plus 1 <license key>
session 1 ip address 10.10.1.1 255.255.255.0
session 1 ip gateway 10.10.1.1
session 1 do setup dns 10.10.1.10 10.10.1.11
session 1 do setup host csc-ssm1.booches.nl
session 1 do setup email-domain booches.nl
session 1 do setup ntfn-email admin@booches.nl
session 1 do setup ntfn-svr-ip 10.10.1.100
session 1 do setup ntfn-svr-port 25
session 1 do setup password <old password> <new password>
session 1 do apply-config

The above commands configure various parameters, like license keys, hostname, IP address, internal mail domain and password credentials.

Define scanning traffic

With the configuration of a service-policy within the Cisco ASA configuration, you define which traffic should be forwarded and scanned by the CSC-SSM modules. The Cisco CSC-SSM modules provide scanning for the following protocols: tcp/http, tcp/smtp, tcp/pop3 and tcp/ftp. The Cisco CSC-SSM modules have their own management interface. This interface is used by the Cisco CSC-SSM modules to check for updates or query the TrendMicro databases for web or e-mail reputation checks. I always exempt the traffic from the Cisco CSC-SSM modules from scanning, because scanning the Cisco CSC-SSM traffic could impact the performance. Below you see and example configuration of the service-policy configuration, including the exemption of the Cisco SSM traffic. The Cisco SSM modules have the IP addresses 10.10.1.1 and 10.10.1.2 in this example.

access-list csc-ssm extended deny tcp host 10.10.1.1 any eq www
access-list csc-ssm extended deny tcp host 10.10.1.1 any eq smtp
access-list csc-ssm extended deny tcp host 10.10.1.1 any eq pop3
access-list csc-ssm extended deny tcp host 10.10.1.1 any eq ftp
access-list csc-ssm extended deny tcp host 10.10.1.2 any eq ftp
access-list csc-ssm extended deny tcp host 10.10.1.2 any eq pop3
access-list csc-ssm extended deny tcp host 10.10.1.2 any eq smtp
access-list csc-ssm extended deny tcp host 10.10.1.2 any eq www
access-list csc-ssm extended permit tcp any any eq www
access-list csc-ssm extended permit tcp any any eq smtp
access-list csc-ssm extended permit tcp any any eq pop3
access-list csc-ssm extended permit tcp any any eq ftp
!
class-map cm-csc-scanning
match access-list csc-ssm
!
policy-map pm-csc-scanning
class cm-csc-scanning
csc fail-open
!
service-policy pm-csc-scanning interface inside

The policy-map uses csc fail-open, which means that all traffic is allowed through if the Cisco CSC-SSM module would (physically) fail and become unavailable. Depending on the security policy used within your organization, you can also configure the Cisco ASA to block all traffic. You should then use the command csc fail-closed.

I configured the service-policy on the inside interface. This is done for logging purposes. When you apply the service-policy to the outside interface, you will only see the outside PAT address in the http logging. Which is fairly useless if you would like to know who is accessing an “illegal” website.

Before placing the Cisco CSC-SSM module in production, be sure that the Cisco CSC-SSM module has access to the internet to query the TrendMicro database. If the Cisco CSC-SSM modules don’t have the appropriate rights to the internet the scanning feature doesn’t work properly and nobody will have access to the internet.

Failover / Synchronization

Cisco ASA appliances are mostly configured in a redundant way, like active/passive or active/active. When both Cisco ASA appliances have their own Cisco CSC-SSM module, you can configure synchronization between the Cisco CSC-SSM modules. This ensures that the configuration of both modules is always the same. I configure failover / synchronization with the following steps.

  1. 1. Completely configure the primary module;
  2. 2. Complete the initial configuration on the secondary module;
  3. 3. Test connectivity between both modules and export the configuration of the primary module;
  4. 4. Open a browser window and enter the following URL in the Address Field: https://<primary IP device address>:8443. The Logon windows appears. Log on, and choose Administration – Device Settings – Device Failover Settings.
  5. 5. Open a second browser windows and enter the following URL in the Address Field: https://<secondary device IP address>:8443. As in step 4, log on and choose Administration – Device Settings – Device Failover Settings.
  6. 6.On the Device Failover Settings window for the primary device, enter the IP address of the secondary device in the Peer IP address field. Enter an encryption key of one to eight alphanumeric characters in the Encryption key field. Click Save, and then click Enable. The following message appears under the windows title: Interscan for CSC SSM could not establish a connection because the failover peer device is not yet configured. Please configure the failover peer device, then try again. This message is normal behavior and appears because the peer is not yet configured.
  7. 7. On the Device Failover Settings window for the secondary device, enter the IP address of the primary device in the Peer IP address field. Enter the encryption key of one to eight alphanumeric characters in the Encryption key field. The encryption key must be identical to the key entered for the primary device. Click Save, and then click Enable. The following message appears under the windows title: InterScan for CSC SSM has successfully connected with the failover peer device. Do not click at anything else at this time for the secondary device.
  8. 8. On the Device Failover Settings window for the primary device, click Synchronize to peer. The message in the Status field at the bottom of the window should state the data and time of the synchronization, for example: Status: last synchronized with peer on 11/01/2010 09:12:12.

Be sure you do not click Synchronize to peer at the end of step 7, while you are still on the Device Failover Settings window for the secondary device. If you do, the configuration you have already set up on the primary device is erased. You must perform manual synchronization from the primary device, as described in step 8. The exception to the auto-synchronization feature is uploading a system patch. A patch must be applied on both the primary and the secondary device.

PIX Failover not working

Today I received the question why a PIX failover configuration wasn’t working. The customer accidentally disconnected the power cable from the primary PIX firewall. The secondary PIX firewall became the active one, but multiple DMZ segments weren’t working anymore. After rebooting the PIX firewall and making that the primary one again, the DMZ segments were reachable again.

I started looking at the failover state of the firewall. Entering the commando show failover on the primary PIX resulted in the following output:

This host: Primary – Active
Active time: 5430735 (sec)
Interface outside (1.1.1.1): Normal
Interface inside (2.2.2.1): Normal
Interface DMZ01 (3.3.3.1): Normal
Interface DMZ02 (4.4.4.1): Normal (Waiting)
Interface DMZ03 (5.5.5.1): Normal (Waiting)
Other host: Secondary – Standby Ready
Active time: 5235 (sec)
Interface outside (1.1.1.2): Normal
Interface inside (2.2.2.2): Normal
Interface DMZ01 (3.3.3.2): Normal
Interface DMZ02 (4.4.4.2): Normal (Waiting)
Interface DMZ03 (5.5.5.2): Normal (Waiting)

Interface DMZ02 and DMZ03 were the problem interfaces.

After doing some simple ping test, I noticed the problem interfaces couldn’t ping each other. Looking at the output above the interfaces are in Normal (Waiting) state. Searching the internet resulted in the following:

Failover does not start to monitor the network interfaces until it has heard the second “hello” packet from the other unit on that interface. This takes about 30 seconds. If the unit is attached to a network switch that runs Spanning Tree Protocol (STP), this takes twice the “forward delay” time configured in the switch (typically configured as 15 seconds), plus this 30 second delay. This is because at PIX bootup and immediately following a failover event, the network switch detects a temporary bridge loop. Upon detection of this loop, it stops forwarding packets on these interfaces for the “forward delay” time. It then enters the “listen” mode for an additional “forward delay” time, during which time the switch listens for bridge loops but not forwarding traffic (and thus not forwarding failover “hello” packets). After twice the forward delay time (30 seconds), traffic resumes flowing. Each PIX remains in “waiting” mode until it hears 30 seconds worth of “hello” packets from the other unit. During the time the PIX is passing traffic, it does not fail the other unit based on not hearing the “hello” packets. All other failover monitoring still occurs (that is, Power, Interface Loss of Link, and Failover Cable “hello”).
Source

Here they are talking about the STP configuration. I couldn’t imagine that the problem is lying in the STP configuration. I started tracking the mac-address on the switches to check the physical patching.

The problem was found directly. The patching of the secondary PIX wasn’t correct. The customer had swapped the two interfaces resulting in patching them in the wrong VLAN. After swapping the patches the failover state functions normally, looking at the output below:

This host: Primary – Active
Active time: 5430735 (sec)
Interface outside (1.1.1.1): Normal
Interface inside (2.2.2.1): Normal
Interface DMZ01 (3.3.3.1): Normal
Interface DMZ02 (4.4.4.1): Normal
Interface DMZ03 (5.5.5.1): Normal
Other host: Secondary – Standby Ready
Active time: 5235 (sec)
Interface outside (1.1.1.2): Normal
Interface inside (2.2.2.2): Normal
Interface DMZ01 (3.3.3.2): Normal
Interface DMZ02 (4.4.4.2): Normal
Interface DMZ03 (5.5.5.2): Normal

This proves again how import correct patching is!!!!!