Welcome to ISAserver.org

WPAD and access failure Q's in EE+NLB

Users viewing this topic: none

Logged in as: Guest

Printable Version

All Forums >> [ISA 2006 Web Proxy] >> General >> WPAD and access failure Q's in EE+NLB

Page: [1]

Message

<< Older Topic Newer Topic >>

WPAD and access failure Q's in EE+NLB - 27.Mar.2008 7:07:30 PM

gbarnas

Posts: 147
Joined: 27.Apr.2005
From: New Jersey
Status: offline

Scratching my head a bit here, because this just doesn't "feel" right..

ISA 2K6 EE, on W2K3-EE, SP2, all current patches, two servers in an array
NIC1 - Internal, NLB, 10.x.y.14 (VIP) .15 (ISA1) and .16 (ISA2), DNS, no gateway
NIC2 - Intra-Array, 172.17.1.1 (ISA1) and .2 (ISA2), no gateway or DNS
NIC3 - External, 192.168.x.y - DMZ subnet behind a Cisco router, GW is router, no DNS

CARP is NOT enabled on any interface (yet).

WPAD is defined in DHCP as "wpad.internal.domain/wpad.dat"
DNS has A records for the hostnames "wpad" and "proxy" pointing to the ISA VIP.

Computers obtain a WPAD.DAT file easily enough via the NLB IP. I manually pointed to "isa1/wpad.dat" and "isa2/wpad.dat" and was surprised to find different results. The core content is the same, but the MakeProxy part of the script only lists the one specific server. I would have expected the VIP, or possibly both servers.

Is this expected?

I'm asking because we just migrated from an ISA 2K array to an ISA 2K6 array, and have experienced a handfull of access issues. We've got about 2000 active sessions, relatively well balanced across the two servers at any given time during the day. A few users have complained about not being able to access sites. Much of thes have been resolved by clearing the user's cache and force-loading the page via Control-F5. A few, however, just will not work. The logs don't show anything being denied, just a request but no response. I'm wondering if NLB has anything to do with it, which is why I looked at the WPAD.DAT files.

There are only 3 user Web access rules -
Permit all users to "host".ups.com - permits the UPS Worldship app to "phone home" without authenticating
Permit HQ staff to Internet - HTTP(S) & FTP (unrestricted)
Permit field staff to Internet HTTP(S) (http filter .EXE, .ZIP, & .RAR)
There are 3 other rules specifically to permit SurfControl to be managed and access its blacklist for downloading - all right out of the SC install guide.
Simple enough that I don't believe the rules are affecting the access. Also, when we test, we test from HQ which has no filtering, and get the same results.

It seems that most of the times we have problems, the site is using PHP. All clients are Web Proxy, set by Auto-Config, controlled by GPO. My team's OU blocks inheritance, so we can manually configure our settings for testing. If I connect to the DMZ subnet (or point to the old ISA proxy), I have no problem accessing these sites. (The old proxy has dozens of rules to work around "access issues" - let's just not go there, OK?)

Any thoughts about the WPAD.DAT differences, or the random site access issues?

Appreciate your time!

Glenn

Post #: 1

Featured Links*

RE: WPAD and access failure Q's in EE+NLB - 27.Mar.2008 7:39:34 PM

Jason Jones

Posts: 2154
Joined: 30.Jul.2002
From: United Kingdom
Status: offline

Hi Glenn,

This behaviour is expected with NLB as the auto configuration script is only aware of the dedicated IP addresses and not NLB VIPs.

This extract from here may help explain things a little:

Failover for Web Proxy Clients in ISA Server 2006 Enterprise Edition

In ISA Server 2006 Enterprise Edition, you have some proxy failover capabilities with client-side CARP capability, and Network Load Balancing (NLB) configuration. Consider the following:

� CARP provides load balancing and cache distribution, but does not provide a true failover solution. For example, Internet Explorer caches the configuration script (Wpad.dat or Isa.routing.script) for 50 minutes by default, and new Web browser sessions will first check the cache for the script. If an ISA Server array member specified in the script becomes unavailable, the client may still try to connect to it with the cached script.

� The configuration script is client-based, and the CARP implementation depends on the client's interpretation of the state of a specific server. This is less resilient to error than an NLB server-based solution.

� Implementing NLB and CARP together provides some failover capabilities by ensuring that the automatic configuration script is highly available. If you have NLB configured, you can specify the NLB cluster's virtual IP address in the location of the automatic configuration script, or by specifying the virtual IP address in the DNS or DHCP WPAD entry. NLB will only forward the request for the script to the available members of the array. The client-side CARP algorithm in the script then ensures that the URL request is handled by the most appropriate array member.

� For true failover capabilities, clients would connect to the array virtual IP address instead of using client-side CARP capabilities in the automatic configuration script.

So, to obtain 'true HA' you need to avoid using the configuration script all together and configure web proxy clients with a specific proxy server definiton which uses an FQDN that resolves to the NLB VIP. In this way you are relying on pure NLB to send the client to the correct proxy server, not the array script. The key drawback with this approach is that you will not benefit from a distributed caching unless you look at enabling server-side CARP.

Not sure if this explains your issues, but should explain the wpad.dat results.

Cheers

JJ

< Message edited by Jason Jones -- 27.Mar.2008 7:42:38 PM >

_____________________________

Jason Jones (MVP)

Silversands Limited http://www.silversands.co.uk
My Blog: http://blog.msfirewall.org.uk/

Get our NEW ISA 2006 Book!: http://tinyurl.com/2gpoo8

(in reply to gbarnas)

Post #: 2

RE: WPAD and access failure Q's in EE+NLB - 27.Mar.2008 7:48:03 PM

Jason Jones

Posts: 2154
Joined: 30.Jul.2002
From: United Kingdom
Status: offline

Some more info: http://blogs.isaserver.org/shinder/2006/06/03/can-carp-and-nlb-support-each-other/

and the "CARP and NLB Integration" section here: http://www.microsoft.com/technet/isa/2006/nlb.mspx

Cheers

JJ

< Message edited by Jason Jones -- 27.Mar.2008 7:51:52 PM >

_____________________________

Jason Jones (MVP)

Silversands Limited http://www.silversands.co.uk
My Blog: http://blog.msfirewall.org.uk/

Get our NEW ISA 2006 Book!: http://tinyurl.com/2gpoo8

(in reply to Jason Jones)

Post #: 3

RE: WPAD and access failure Q's in EE+NLB - 27.Mar.2008 8:50:21 PM

gbarnas

Posts: 147
Joined: 27.Apr.2005
From: New Jersey
Status: offline

Thanks - I had been reading that earlier today, but had too much going on for it to absorb clearly. :o The thing that had me puzzled is that the wpad.dat from ISA 2K system included both addresses.

I've been hesitant to enable CARP, based on what I'm reading and the need to define exclusions. Our field staff uses lots of web-based applications during the day, and I'm concerned that I may be opening a can of worms. I've already had to define some cache exclusions when people at one office were getting user IDs of people at another office from a commonly used web app.

If I wanted to distribute the wpad.dat with the array VIP, could I not just take the file from one of the ISA servers, change the server address to the VIP, and publish it on any available IIS server, such as our Intranet server? Or do I need to look at custom-writing the config file to not do client-side CARP if/when I enable that?

Thanks again,

Glenn

(in reply to Jason Jones)

Post #: 4

RE: WPAD and access failure Q's in EE+NLB - 28.Mar.2008 5:29:49 AM

Jason Jones

Posts: 2154
Joined: 30.Jul.2002
From: United Kingdom
Status: offline

By using the auto configuration script, you realise you are already using client-side CARP - yes? Enabling CARP within the network properties in the GUI is talking about server-side CARP.

Yeah, you can host the WPAD.DAT elsewhere and use the VIP but I think this will break distributed caching as NLB will just balance the requests irrespective of where the item may have been cached previously. The net result will be that your cache hit ratio will likely be worse and content will be duplicated in the cache directories on both servers.

From my understanding, it is often a call between performance or high availability. If you want "true HA" you need to go with NLB and not use client-side CARP. If you want best performance you use client-side CARP (auto config script) but you accept that NLB is only providing fault tolerance for the script as the clients will use the array member dedicated addresses in the script, not the VIP.

There is a half-way house (as I hinted at above) where you use NLB without client-side CARP, but enable server-side CARP. The downside to this is that the array members end up running the script on behalf of users (to find out which array member has the cached content) and hence this can have a performance hit on the servers. It is always more efficient for the clients to do the script processsing, but as you have seen this has HA limitations.

I have used all three scenarios and it normally comes down to which elements is more important, HA or performance.

Cheers

JJ

_____________________________

Jason Jones (MVP)

Silversands Limited http://www.silversands.co.uk
My Blog: http://blog.msfirewall.org.uk/

Get our NEW ISA 2006 Book!: http://tinyurl.com/2gpoo8

(in reply to gbarnas)

Post #: 5

RE: WPAD and access failure Q's in EE+NLB - 28.Mar.2008 11:18:56 AM

gbarnas

Posts: 147
Joined: 27.Apr.2005
From: New Jersey
Status: offline

Yes, I get the client side CARP concept. I was referring to enabling the Server-side. Too many chainsaws in the air this week to properly absorb the info on setting this up (or properly describe my thoughts, it seems). I'm very comfortable with most of this, but am having trouble wrapping my mind around the client configuration process, and wpad.dat specifically with enabling server-side CARP.

Our ISA servers are brand new dual dual-core Xeons with 4G RAM. We have about 3000 users, with an average load of 1100-1200 users per ISA server. With this load we rarely peak above 12%, and most of the time they are idling between 4-6%. Internal Gbit NIC peaks at about 1.25%, and the External 100Mb has peaked at 10%, so I think I've got plenty of headroom to enable server-side CARP.

So - this is where I think I'm at -
I manually create a WPAD.DAT file that references the NLB VIP instead of specific ISA hosts; (I need to find some good docs/examples - my mission for today)
I host that wpad.dat file on our Intranet server (cluster);
I enable server-side CARP on the ISA Internal network;
I change the DNS CNAME record for "wpad" to point to the Intranet server VIP instead of the NLB VIP;

Have I missed anything? I've intentionally not disabled the ISA wpad publishing because I am pretty sure I can script the creation of the "real" wpad.dat to automatically include the addresses and domain exclusions defined by ISA. My real hangup at the moment - can I base the wpad.dat file on the one that ISA generates? Do I need to merely remove CARP definitions?

Thanks again!

(in reply to Jason Jones)

Post #: 6

RE: WPAD and access failure Q's in EE+NLB - 28.Mar.2008 1:23:36 PM

Jason Jones

Posts: 2154
Joined: 30.Jul.2002
From: United Kingdom
Status: offline

The following article may help with syntax etc:

http://www.isaserver.org/tutorials/Load-Balancing-Web-Proxy-Clients-With-ISA-Server-2004-Standard-Edition-Part1.html

However, if you don't need the client-side CARP element, why do you need the auto configuration script at all? Surely you can achieve the same thing with GPOs to simply define the proxy server FQDN and exceptions? The benefit of an ISA hosted WPAD.DAT is that it automatically updates it from the ISA GUI. If you move the WPAD.DAT file you lose these benefits anyhow, so you may as well not use one at all IMHO. Does this make sense?

This way NLB will provide true HA and server-side CARP should solve the distributed caching problem...no script needed and much simpler overall I guess...

I also think you need to enable server-side CARP on the intra-array network if this is how the array member intra-array communication addresses are defined (not sure if you have dedicated intra-array networks with dedicated NICs). There has always been a lot of contradictory info on whether you enable it on the Internal network OR the intra-array network (or both!) and I have never got it nailed down...but I *think* I have used intra-array if the network is present (and defined inthe intra-array comms addresses) and Internal if is has not (e.g. intra-array comms addresses are the internal IP addresses).

Be interested to hear your thoughts and see how you get on

Cheers

JJ

< Message edited by Jason Jones -- 28.Mar.2008 1:26:27 PM >

_____________________________

Jason Jones (MVP)

Silversands Limited http://www.silversands.co.uk
My Blog: http://blog.msfirewall.org.uk/

Get our NEW ISA 2006 Book!: http://tinyurl.com/2gpoo8

(in reply to gbarnas)

Post #: 7

RE: WPAD and access failure Q's in EE+NLB - 28.Mar.2008 2:05:43 PM

gbarnas

Posts: 147
Joined: 27.Apr.2005
From: New Jersey
Status: offline

:D
We're still sorting out the GPO mess from the ex-ex-admin (1200 GPOs???) We're down to a few, and a totally new (sane) GPO deployment upcoming, so I don't want to rock the boat with new requests at the moment. The Proxy Auto-Config is already in the GPO, and our DNS/DHCP are configured & working.

From what I have read, CARP is enabled on the Internal network (where the requests come in) and you enable Web Proxy Requests on the Intra-Array network so the array members can make proxy queries to each other. I forgot to mention that last part (probably while dodging a chain saw) in my prior post, but assume it will go hand-in-hand with enabling CARP on the Internal network. Yes - we do have a dedicated intra-array network.

I finally found a somewhat dated but comprehensive explaination of the WPAD script format (http://wp.netscape.com/eng/mozilla/2.0/relnotes/demo/proxy-live.html) so that's making it easier to think about the approach.

My plan is to leave the ISA server auto-config script in place (maybe change the port to make it less accessible). On the Intranet server, I'll run a script that does a GET to the ISA server and grabs the WPAD.DAT file. It will have a template for the wpad file I want, and will copy the declarations from the ISA script to the production script. This way, I eliminate the CR - er - CARP from wpad, but still have the benefit of auto-config for network address and domain exclusions.

I'm going to write the script and set my hosts file to point wpad to the Intranet server (to get my custom script) and test it against our QA proxy. I'll let you know how this works out. Thanks for your insights!

Regards,

Glenn

(in reply to Jason Jones)

Post #: 8

RE: WPAD and access failure Q's in EE+NLB - 28.Mar.2008 5:56:50 PM

gbarnas

Posts: 147
Joined: 27.Apr.2005
From: New Jersey
Status: offline

Jason,

Just a quick follow-up. The other mystery has been solved. The problem with random errors - mostly 10061 - has been traced to the edge firewall, which only permits 80 and 443 outbound. The problematic sites were going to alternate ports on some pages for some odd reason. (bad design?) :o

Now need to convice the networking group that ISA's protocol filtering will be superior to port blocking, and we should permit the ISA array to go out on all ports.

Glenn

(in reply to gbarnas)

Post #: 9

RE: WPAD and access failure Q's in EE+NLB - 28.Mar.2008 8:11:50 PM

Jason Jones

Posts: 2154
Joined: 30.Jul.2002
From: United Kingdom
Status: offline

quote:

ORIGINAL: gbarnas

:D
We're still sorting out the GPO mess from the ex-ex-admin (1200 GPOs???) We're down to a few, and a totally new (sane) GPO deployment upcoming, so I don't want to rock the boat with new requests at the moment. The Proxy Auto-Config is already in the GPO, and our DNS/DHCP are configured & working.

From what I have read, CARP is enabled on the Internal network (where the requests come in) and you enable Web Proxy Requests on the Intra-Array network so the array members can make proxy queries to each other. I forgot to mention that last part (probably while dodging a chain saw) in my prior post, but assume it will go hand-in-hand with enabling CARP on the Internal network. Yes - we do have a dedicated intra-array network.

I finally found a somewhat dated but comprehensive explaination of the WPAD script format (http://wp.netscape.com/eng/mozilla/2.0/relnotes/demo/proxy-live.html) so that's making it easier to think about the approach.

My plan is to leave the ISA server auto-config script in place (maybe change the port to make it less accessible). On the Intranet server, I'll run a script that does a GET to the ISA server and grabs the WPAD.DAT file. It will have a template for the wpad file I want, and will copy the declarations from the ISA script to the production script. This way, I eliminate the CR - er - CARP from wpad, but still have the benefit of auto-config for network address and domain exclusions.

I'm going to write the script and set my hosts file to point wpad to the Intranet server (to get my custom script) and test it against our QA proxy. I'll let you know how this works out. Thanks for your insights!

Regards,

Glenn

Sounds like a good plan and I understand your reservations with GPOs

Look forward to hearing how it goes...

_____________________________

Jason Jones (MVP)

Silversands Limited http://www.silversands.co.uk
My Blog: http://blog.msfirewall.org.uk/

Get our NEW ISA 2006 Book!: http://tinyurl.com/2gpoo8

(in reply to gbarnas)

Post #: 10

RE: WPAD and access failure Q's in EE+NLB - 28.Mar.2008 8:16:27 PM

Jason Jones

Posts: 2154
Joined: 30.Jul.2002
From: United Kingdom
Status: offline

quote:

ORIGINAL: gbarnas

Jason,

Just a quick follow-up. The other mystery has been solved. The problem with random errors - mostly 10061 - has been traced to the edge firewall, which only permits 80 and 443 outbound. The problematic sites were going to alternate ports on some pages for some odd reason. (bad design?) :o

Now need to convice the networking group that ISA's protocol filtering will be superior to port blocking, and we should permit the ISA array to go out on all ports.

Glenn

Ah, that makes sense...the fact that ISA can provide filtering based upon user and not IP is normally a good reason to let ISA do the hard work compared to the edge firewall

So with this sorted, you just need to find the optimum design for the stuff we talked about above...as you can tell, it is not always easy to have the best of both worlds with performance and HA, especially out of the box

I think it would be good NLB and client-side CARP were somehow integrated such that the script could provide true HA when NLB is enabled...maybe in the next version

_____________________________

Jason Jones (MVP)

Silversands Limited http://www.silversands.co.uk
My Blog: http://blog.msfirewall.org.uk/

Get our NEW ISA 2006 Book!: http://tinyurl.com/2gpoo8

(in reply to gbarnas)

Post #: 11