Connecting the world…

reverse

Cisco ASA & ESX: strange ARP behavior

Last week I had a very strange problem with a Cisco ASA firewall. The firewall is configured with multiple interfaces, including a DMZ interface. There are multiple servers in the DMZ. These servers are physical and virtual servers. The virtual servers are VMware servers in a blade environment.

I configured the feature

ip verify reverse-path interface DMZ

to prevent spoofing to occur. I also configured a transparent static NAT rule between the Inside network and the DMZ network and multiple static NAT rules between the DMZ network and the Outside network. I left the proxy ARP feature configured with its default settings.

The customer was complaining about log in problems and connectivity problems on the DMZ servers, especially between different DMZ servers. I have done some research and noticed that all problems were related to DMZ servers in the blade environment.

I started some connectivity test and noticed some strange ICMP behavior on the specific servers. When I started a ping from one DMZ VMware server to an other DMZ server on the same ESX host, the first ping responded with an echo-reply, but consequent pings failed. Looking at the ARP table of the server, I noticed that the firewall responded with its own MAC address for every ARP broadcast.

Looking at different forums on the Internet, everybody is speaking about the proxy ARP feature and that you should disable this feature. By default proxy ARP is enabled and I always leave it enabled. Till now I never had this problem. After disabling the proxy ARP feature for the DMZ interface

sysopt noproxyarp DMZ

the problem was solved, because the firewall doesn’t respond to the ARP queries, except for its own interface. Digging a bit deeper on forums, I never found one thread who explains why the proxy ARP feature should be disabled to solve this particular problem.

In my opinion this problem is related to the VMware environment, because I don’t have these problems with physical DMZ servers. So it is strange why the DMZ servers on the same ESX hosts cannot see each other and why does the firewall respond to the ARP queries?

In the near future the blade environment (ESX hosts, network configuration and SAN configuration) is changed, so I hope to find the exact cause and solution of the problem. Does anybody else have some suggestions??

Change password through LDAPS on ISA server

Today I received the question about allowing users to changes his/her password through webmail, whereby webmail is published via an ISA server 2006 reverse proxy. This is possible, but it requires the configuration of LDAPS to authenticate users.

I started by configuring a Certificate Authority (CA) on a member server in the domain. During the installation of CA a root certificate is generated. You need to export this root certificate with private key. Next I imported the certificate on the reverse proxy server, but didn’t mark the private key as exportable. So the root certificate cannot be exported from the reverse proxy server with its private key in the future. I checked if binding to the Active Directory is possible by using the tool ldp.exe.

The last part is configuring LDAP Validation in ISA. Go to Configuration –> General –> Specify RADIUS and LDAP Servers. First you need to add a LDAP server set, like shown in the following picture.

Important when configuring the LDAP server set is the usage of the FQDN as LDAP server hostname. This FQDN should be exactly the same compared to the FQDN mentioned in the imported root certificate.

The last step is configuring the LDAP server mapping, which is also shown below.

Because I don’t want to add a domain name during the login procedure on the OWA login page, like DOMAIN\USER, I use the Login Expression wildcard character * and link that to the configured LDAP server set. Now you can login with just username and password, instead of domain\username and password.

Next I configure a OWA Publishing policy like always, but on the Listener I use LDAP as authentication mechanism. On the Listener Forms tab you can enable or disable the options:

  • Allow users to change their passwords;
  • Remind users that their passwords will expire in this number of days;

These options add some extra option to the OWA login page. Another step to configure is the allowed users. In most environments I use the group Domain Users as allowed OWA group, because mostly all users are allowed to use OWA, else you need to configure a separate user group in Active Directory. On the Users tab you remove the All Authenticated Users and click Add. You need to define a new user group, like shown below.

User Group

This means that if you are member of the group Domain Users, you are allowed to use OWA.

The last step is configuring the public path. When logged in to OWA, you have the option to change your password through the options page. To use this feature, you need to added another path to the Path configuration in the reverse proxy server. The path, which should be added, is /iisadmpwd/*, where the External Path is the same as the Internal Path.

Over at isaserver.org, Thomas Shinder wrote a great post about using LDAPS with OWA and multiple domains. The article is called LDAP Pre-Authentication with ISA 2006 Firewalls: Using LDAP to Pre-Authenticate OWA Access.

Automatic Log In Reverse Proxy with FBA

Recently I configured another ISA 2006 server as reverse proxy to publish the Exchange 2007 OWA environment on a secure way to the internet. The customer where I configured the reverse proxy is migrating from Novell GroupWise to Microsoft Exchange.

During the migration period, the customer has specific requirements when connecting to the webmail environment from the internet. The customer is using GroupWise and OWA next to each other. He build an own log in page for webmail. The user enters this credentials, which are check against a directory with the help of LDAP. Depending on the LDAP group the user is member of, he is redirected to GroupWise webmail or OWA.

The problem was that when the client gets redirected to OWA, he had to provide his credentials a second time to log in to OWA. The customer wants to use Single Sign On, so the user credentials should be automatically posted on the OWA sign in page. The customer has the option to edit his own sign in page, so specific parameters can be added when redirecting the client to the ISA 2006 reverse proxy.

I had to figure out, which URL is posted to the ISA reverse proxy after the user credentials are entered in the web browser. I started to search the internet and different forums, but the solution wasn’t found. So there was nothing else left, than starting the good old sniffer and capture the complete process. I always use WireShark as sniffer, which is a very powerful tool. I added a filter so only the traffic to and from the ISA 2006 server is captured.

I started the sniffer, logged in on the ISA 2006 server, stopped the sniffer and started analyzing the sniffer results. After some digging I found the POST statement, which is displayed below.

POST-STATEMENT

Looking at the POST statements, I now knew the exact URL which is posted to the ISA 2006 server. I copied the URL to notepad, changed the username and password to another account. Copied the new URL and pasted that one in my browser and HIEHOE. After hitting enter, I don’t get any sign in page, but get directly to my webmail. So exactly as the customer wants.

The default POST URL has the following format:

https://<DNS name>/CookieAuth.dll?Logon?curl=Z2Fowa&flags=0&forcedownlevel=0& formdir=1&trusted=0&username=<username>&password=<password>
&SubmitCreds=Log+On

ISA 2006 Authentication over HTTP

I implemented different ISA 2006 Reverse Proxy servers in conjunction with Microsoft Exchange 2003 or Windows Exchange 2007.

Today I configured ISA 2006 with Exchange 2007. I configured the Reverse Proxy server as I did always. And the connection from outside the network works perfectly. On the internal Exchange server I configured Basic and Integrated Authentication on the OWA virtual directory. The problem is that internal users now automatically log in to their webmail box when entering the URL from the Exchange server.

This is not the desired configuration, because internal users should be able to open other people’s mailboxes by logging in as that user. The customer also has an ISA 2006 on the internal network for forwarding proxy purposes.

I decided to publish Exchange 2007 on the internal ISA 2006 server as well. The configuration should use Form Based Authentication (FBA) over HTTP. After configuring and trying the connection, the user can’t access the ISA logon page. In the logging you find that Authentication over HTTP isn’t allowed.

Error Code: 403 Forbidden. ISA Server is configured to block HTTP requests that require authentication. (12250)

This is a default setting in ISA 2006 which can be disable. To allow Authentication over HTTP go to the Listener configuration. Go to the Authentication tab and Select Advanced. In the next tab enable the option Allow client authentication over HTTP. This option enables the using FBA over HTTP.

PDA Active Sync – Invalid Certificate

The usage of Pocket PCs (PDAs) becomes more and more a default feature for business. The last months I have installed quit some Windows ISA 2006 servers for Reverse Proxy purposes. I have installed them normally for webmail only, but lately I have added the Microsoft Active Sync feature.

The Pocket PCs connect to the organization via UMTS, GPRS, USB with laptop or whatever with an internet connection. Today I had the same job on the schedule: Enable Active Sync for Pocket PCs.

I thought by myself: EASY JOB, but NOT. After configuring the ISA reverse proxy I used a Pocket PC emulator to test the Active Sync features. I received the following error message when synchronizing:

pda

I found this a strange message, because clients use the same URL as the Pocket PC for accessing their webmail and they never receive an error message for an untrusted certificate.

The used certificated is issued by Equifax Secure Global eBusiness CA-1. This is a common and one of the better CA’s.

I had to dig deeper into the problem. I tried to install the certificate on the Pocket PC, but no luck. I searched the internet and found a tool called Microsoft Exchange Server Disable Certificate Verification. You can find an executable here, which can be used when using the Pocket PC in conjunction with a PC through USB. I also found a similar tool to install on the pocket PC, this is called AS_Cert_OFF.cab. The tool wasn’t the solution to the problem, so I had to dig deeper.

I was thinking way to complex, the problem was fixed by requesting a new certificate. The used certificate didn’t support Pocket PC. Comparing the different SSL certificates on QuickSSL.com I noticed I had to use a QuickSSL Premium certificate. This certificate supports popular mobile devices and smartphones.

After generating a CSR, requesting the certificate and installing the certificate on the ISA server, the connection and synchronization works like a charm. At least for the most PDA’s. Some PDA’s received the following error 80072f7d. After searching some forums, I found the solution in adding a registry key. I added the following registry key:

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows CE Services]
“AllowLSP”=dword:00000000

After adding the key to the registry, all Pocket PC’s synchronized perfectly.