• RSS
  • Twitter
  • FaceBook

Welcome to ISAserver.org

Forums | Register | Login | My Profile | Inbox | RSS RSS icon | My Subscription | My Forums | Address Book | Member List | Search | FAQ | Ticket List | Log Out

Web Chaining Problem

Users viewing this topic: none

Logged in as: Guest
  Printable Version
All Forums >> [ISA Server 2004 Firewall] >> General >> Web Chaining Problem Page: [1]
Login
Message << Older Topic   Newer Topic >>
Web Chaining Problem - 23.Jun.2007 8:31:25 PM   
Juliang

 

Posts: 2
Joined: 23.Jun.2007
Status: offline
I am using ISA Server 2004 SP2 with HTTP filter (KB916106) hotfix installed and trying to configure a web chaining rule to redirect all web (HTTP and HTTPS) requests to a 3rd party upstream proxy server which implements web content filtering for my school. My setup is as follows:

Internal Network-> My ISA Server -> 3rd party content filter -> External Internet

With a web Chaining Rule as follows:
To: External
Action: Redirect to upstream server x.x.x.x

Plus necessary access rules for HTTP and HTTPS.

When the web chaining rule is enabled and computers are either SecureNAT or Firewall clients of the ISA server they cannot browse to some websites. These clients receive the following types of problematic HTTP pages instead:

-The correct page but with missing pictures and messed up layout.
-Web sites loading the home page of their hosting provider instead of their own page.
-Web sites stating that there is not website hosted at the specified url.

When the web chaining rule is disabled web proxy clients can browse the internet normally. The only client type that works with the web chaining rule enabled is the web proxy client, which can browse the internet normally with the web chaining rule either enabled or disabled. I believe this problem is due to ISA server not forwarding host headers to the upstream proxy.

The symptoms of this problem can be illustrated by using www.google.co.uk as an example:

If open up internet explorer configured as a web proxy client on a workstation on the internal network and browse to www.google.co.uk under the logging tab on the isa console I can see that the request is indeed forwarded to the upstream server and is allowed by my “Web” access rule. I notice that the URL field contains http://www.google.co.uk/ and the client browser receives the correct Google UK home page. However if I disable the proxy server in internet explorer (the workstation falls back to a SecureNAT client) and browse to www.google.co.uk again the logging tab displays http://209.85.135.104/ instead. That’s fair enough though because ISA doesn’t log URLs for non web proxy clients but the client doesn’t receive the same Google page! The client browser receives the same page as if they had typed 209.85.135.104 into the browser, the Google English (not UK) home page.

All of the above is not a problem when the web chaining rule is disabled or clients are configured as web proxy clients. I do not think this is a problem with my setup because with the web chaining rule disabled all three ISA client types work correctly. Also I do not think it is a problem with the upstream proxy because it works just fine with web proxy clients.

I seen this problem described in the following posts:

http://forums.isaserver.org/Web_chaining_breaking_-_but_not_when_proxying_via_8080/m_2002027345/tm.htm

http://forums.isaserver.org/Problem_with_upstream_Proxy_server/m_2002032396/tm.htm

So far the only suggestion is make clients web proxy clients but what if this isn’t an option? Are SecureNAT and Firewall clients NOT supported with web chaining?

I hope someone can shed some light on this!

Many thanks
Julian Gilbert
Post #: 1
RE: Web Chaining Problem - 15.Aug.2007 2:43:52 PM   
wasserja

 

Posts: 56
Joined: 4.Dec.2002
Status: offline
I am having this same issue, but I haven't found the fix yet.  Does anyone else have a solution?

(in reply to Juliang)
Post #: 2
RE: Web Chaining Problem - 10.Dec.2007 1:07:25 PM   
ferrix

 

Posts: 547
Joined: 16.Mar.2005
Status: offline
My guess is that this is a convergence of 3 behaviors. 
1)
ISA sends the URL as http://IP/path. 
2)
your upstream proxy may be choosing to forward the URL on in the complete format, i.e.
GET http://IP/path
Host: thehost

instead of
GET /path
Host: thehost

3)
the web server may be honoring the URL hostname and ignoring the host header.

If ISA was the upstream proxy item 2 wouldn't occur since it always sends the hostless form of the URL line to target web servers.

Can you wireshark the traffic emitted from the upstream proxy, or is it not in your realm of control?  It would be interesting to know what the ultimate request looked like.

(in reply to wasserja)
Post #: 3
RE: Web Chaining Problem - 10.Dec.2007 2:25:32 PM   
jgilbert

 

Posts: 9
Joined: 15.Apr.2005
From: U.K.
Status: offline
Unfortunately the upstream proxy is not under my control but I can try and request that information!

I can however run a data capture on my ISA server’s external interface and interestingly I found that for a Web Proxy client my server sent the following to the upstream proxy:

GET http://www.google.co.uk/
HOST: www.google.co.uk

and for a SecureNAT client:

GET http://209.85.135.147/
HOST: www.google.co.uk

My ISA server is sending the URL in complete format, is this normal behaviour?

(in reply to ferrix)
Post #: 4
RE: Web Chaining Problem - 10.Dec.2007 2:30:10 PM   
ferrix

 

Posts: 547
Joined: 16.Mar.2005
Status: offline
Yes that is normal for ISA and arguably acceptable in general.  When forwarding to a proxy a full http://URL format *must* be used.  The argument is whether ISA "should" assume the Host header is correct and use it on the URL line, or just use the IP like it does.

The remaining unknown is what the upstream proxy is doing with that request, i.e. what the req to google looks like.  That is important in understanding the ultimate pathology.

(in reply to jgilbert)
Post #: 5
RE: Web Chaining Problem - 10.Dec.2007 2:42:31 PM   
jgilbert

 

Posts: 9
Joined: 15.Apr.2005
From: U.K.
Status: offline
Yep that makes sense to me since the same request sent to www.google.co.uk with web proxy chaining turned off looked like this:

GET /
HOST: www.google.co.uk

(in reply to ferrix)
Post #: 6
RE: Web Chaining Problem - 6.Jun.2008 3:25:19 PM   
jgilbert

 

Posts: 9
Joined: 15.Apr.2005
From: U.K.
Status: offline
I have been able to get a capture of a packet leaving the upstream proxy and going to a web server, it is in the form:

GET /
HOST: 209.85.135.147


I have also been told that the web filtering is being carried out by a squid proxy and a filter based on Dans Guardian. The source code for Dans Guardian shows that it will ignore the HOST value if the GET is an absoluteURI. The following is commented in dans guardian’s source code:

quote:

A request may be in the form:

GET http://foo.bar:80/ HTML/1.0 (if :80 is omitted 80 is assumed)
or:
GET / HTML/1.0
Host: foo.bar (optional header in HTTP/1.0, but like HTTP/1.1, we require it!)


The actual code shows that if Dans Guardian sees a request in the form of


GET http://209.85.135.147/
HOST: www.google.co.uk


it will determine the hostname as 209.85.135.147 and ignore www.google.co.uk. I do not know what squid does but according to RFC 2616 section 5 a web server MUST ignore the HOST value if an GET absoluteURI is sent, therfore I think squid is following this. Nowhere that I can see in RFC 2616 is there a requirement for a proxy to use the HOST value.

Does anyone have experience of using ISA while web chaining to an upstream squid proxy, particularly with ISA as a transparent proxy?

(in reply to jgilbert)
Post #: 7
RE: Web Chaining Problem - 6.Jun.2008 6:02:03 PM   
ferrix

 

Posts: 547
Joined: 16.Mar.2005
Status: offline
By my reading of the RFCs, your upstream proxy is violating spec by rewriting the host header, not "ignoring" it.  But it's an arguable case because of how ISA (uniquely) constructs the URL by using the IP.

A web filter could make the ISA side compatible by rewriting the outbound URL line so it uses the host header as the server name instead of the IP address. 

(in reply to jgilbert)
Post #: 8
RE: Web Chaining Problem - 7.Jun.2008 10:07:34 AM   
jgilbert

 

Posts: 9
Joined: 15.Apr.2005
From: U.K.
Status: offline
I tested using Squid/2.5.Stable9 proxy server and found the following:

A request sent from a client to the squid proxy in the form

GET http://my.dns/
HOST my.differentdns


Generates a GET request from the squid proxy in the form

GET /
HOST my.dns


And a request in the form

GET http://209.85.135.147/
HOST: www.google.co.uk


Generates a GET request in the form

GET /
HOST: 209.85.135.147


This proves that it is squid which is rewriting the HOST value and therefore ISA Server Web Chaining is NOT compatible with an up stream squid proxy.

(in reply to ferrix)
Post #: 9

Page:   [1] << Older Topic    Newer Topic >>
All Forums >> [ISA Server 2004 Firewall] >> General >> Web Chaining Problem Page: [1]
Jump to:

New Messages No New Messages
Hot Topic w/ New Messages Hot Topic w/o New Messages
Locked w/ New Messages Locked w/o New Messages
 Post New Thread
 Reply to Message
 Post New Poll
 Submit Vote
 Delete My Own Post
 Delete My Own Thread
 Rate Posts