ISA Gurus, help please. I'm in panic and do not know what to do/
We have a branch office connected to hq by VPN (IPSec tunnel) Both at hq and the branch we have ISA 2006 SP1. The trouble is thet sometimes while connecting by RDP protocol (terminal server in hq) branch ISA, or hq ISA drops RDP packets with 0x80070008 error NOT ENOUGH MEMORY. But both ISA servers had more than 200Mb free physical memory.
I Have the same issue with dhcp relay packets. I Have ISA 2006 Enterprise whith dhcp relay configured no the routing and remote access service. I have a few Branches that are all connected with IPVPN and they all come to the ISA through the same network/interface. On another network/interface I have the servers segment with the dhcp server, configured with Superscopes. All the branches works fine except one. When a client tries to get an IP address from the DHCP server, through the dhcp relay that resides on the ISA machines I get errors like 0x80070008 ERROR_NOT_ENOUGH_MEMORY. What could it be ? I remind you that I have something like 10 more branches on the same network on the same side of the ISA. all working just fine..
No, it's an IPVPN network, Cisco based. All the routers are configured to relay the dhcp request to the ISA and the ISA forward the request to the DHCP. It works perfectly with all the braches except the one I get those errors in the ISA log. When I look at the log records for the other branches I can see very clear the packet going through the ISA to the DHCP and back. When I look at that specific branch I also see the packets coming through the ISA to the DHCP but when the DHCP sends the packets back I get those errors in the ISA log.
I have the same error with L2TP VPN. VPN connections are diconnected periodically with error: Denied Connection; result_code = 0x80070008 ERROR_NOT_ENOUGH_MEMORY; protocol = IP IPsec NAT-T Client; direction = vpn_clinet -> server.
Software configuration: Windows 2003 server SP2 ISA2006 Standart SP1
(First access rule) I have standart VPN L2TP system access policy where IPsec NAT-T Client rule is: port = UDP:4500; direction = Send Recieve, from external to localhost.
(Second access rule) And i have another access rule in Firewall Policy Rules: port = UDP:1-65535; direction = Send, from localhost to external.
This is last rule and maybe strange behavior ISA2006 are reason of my troubles.
Strange behavior becouse, first VPN L2TP rule should be processed before all firewall policy rules as for sending IPsec NAT-T, as for recieving IPsec NAT-T.
However, the diagnostic log showed me that sending IPsec NAT-T packet processed by the first rule, but recieving IPsec NAT-T processed by the second rule.
I think that ISA2006 dont destroy not finished send-recieve udp pair, and new udp packet from external create a new record in memory for new pair. As result we have not enough memory.
I disable my second rule, and now LT2P VPN work fine.