Just had a fun morning trying to figure out a very strange problem.
ISA server 2006 with two NICs, one in a DMZ another internal. Using NLB. Sinlge ISA node though. Hope to add a second node in future but may be on new systems before then.
Anyway. Last night, our telcom team pulled the connection from the Cisco firewall that manages the external/DMZ/internal. They pulled the DMZ connection between the firewall and some DMZ Cisco switches. All the DMZ servers are connected to these switches, including the ISA server.
They then put our core internal router inline between the firewall and the DMZ switches.
After the change, the ISA server was suddenly not able to pass any traffic. There are rules for OWA, POP, SMTP, RPC/HTTPS, etc.
I was able to RDP to the ISA server and after some investigations and suggestions from the event logs, I restarted the MS Firewall Service. This caused the ISA server interfaces to hang and I could not reconnect my RDP session. I got on the console and decided to disable and enable both interfaces.
This seemed to fix the HTTP/HTTPS traffic, but the POP and SMTP would not work. I ran some pcaps on the inside interface and I noticed that all POP3 and SMTP was going from the DMZ interface to the SMTP/POP3 internal server, but seemed to be crossing the ISA server (if that makes sense).
I had to do a full reboot of the ISA server to get all working again.
What might have triggered ISA to start sending POP and SMTP from it's DMZ IF to the internal NIC?
And yes I have correct persistent routes in place.