SmartFilter Internet Database Download (Full Version)

All Forums >> [ISA Server 2004 Misc.] >> 3rd Party Add-ons



Message


PaulLinton -> SmartFilter Internet Database Download (23.Jan.2009 12:35:44 PM)

Hello All!

I really hope you can help me - I'm completely new to ISA. 

Our network environment is all Windows 2003 Standard SP1 servers and Windows XP Pro SP2 clients.  Currently, I have our ISA 2004 server working as a Web Proxy only; it is configured to require all users to authenticate using the Integrated option.  Recently I started investigating content filtering services and have installed the Secure Computing's SmartFilter (version 4) as a companion product for our ISA 2004 server.

One problem I have with it is the need for the Internet Database Download to run through Basic authentication.  Due to the security needs of our environment I cannot have Basic authentication (where passwords can be easily sniffed) running on the server.  As an alternative, the support rep said to build a firewall rule that would allow the internet database to download.  Unfortunately my efforts on this seem to be failing so far. 

Here are the options for the rule I am currently trying:
Name:       Allow InternetDatabase Download
Action:      Allow
Protocols:  HTTP
From:        Local Host
To:            URL Set (SmartFilter address)
Condition: All Users

I have placed this as my top priority Firewall rule.

Anyone have ideas on how to get this working?





richardhicks -> RE: SmartFilter Internet Database Download (23.Jan.2009 12:58:40 PM)

Hi Paul,
 
Make sure in your Smartfilter software that you DON'T specify a proxy server or credentials.  Also, double check to see that you are using HTTP to download the database and not HTTPS. 




PaulLinton -> RE: SmartFilter Internet Database Download (23.Jan.2009 1:40:34 PM)

Hello Sir,

Thanks for looking at this so quickly.  I opened the SmartFilter Admin Console, then went into Enterprise Settings > Download Setup and removed the proxy server info.  Then I went into the ISA Plugin > Set Advanced Options > Download Setup and removed the proxy server info.  Deployed the changes and hit the "Download Internet Database" button ... it failed again.

When I checked download type is HTTP in both areas.

On the ISA side, I have the following on the URL set:
http://list.smartfilter.com
list.smartfilter.com

(I thought maybe dropping the http:// from it may have an effect on its ability to download)

Any other thoughts?




richardhicks -> RE: SmartFilter Internet Database Download (23.Jan.2009 1:46:32 PM)

You might want to try using a domain name set as opposed to a URL set.  Also, do you have more than one ISA firewall here?  Or is this a single server?




PaulLinton -> RE: SmartFilter Internet Database Download (23.Jan.2009 2:03:16 PM)

I like the way you're thinking.  This is a single ISA server.  Here are the results of my latest test:
I added "list.smartfilter.com" to the "system policy allowed sites" domain name set and I changed the rule to the following:

Name:       Allow InternetDatabase Download
Action:      Allow
Protocols:  All Outbound Traffic
From:        All Networks (and Local Host)
To:            Domain Name Set (System Policy Allowed Sites)
Condition: All Users, User Set: Secure Computing (Domain Account)

Then, I went into the SmartFilter Admin Console and tried again.  Still not working.  Will a rule of this type bypass the Authentication requirement I setup in the ISA management console (Configuration > Networks > Internal > Web Proxy > Authentication)?  Am I approaching this situation from the wrong angle (if so, what is a better way)?

By the way, I'm really appreciative of your advice so far.  Please continue with your suggestions.

Thanks!




richardhicks -> RE: SmartFilter Internet Database Download (23.Jan.2009 2:21:21 PM)

Hi Paul,
 
quote:

Condition: All Users, User Set: Secure Computing (Domain Account)

 
I'm a little confused by this statement.  The access rule should apply to 'all users' only.  There shouldn't be any other users or groups specified in the access rule.
 
quote:

Will a rule of this type bypass the Authentication requirement I setup in the ISA management console (Configuration > Networks > Internal > Web Proxy > Authentication)? 

 
Since the request in this case is coming from the local host, the authentication settings you specified for the web proxy listener on the Internal network won't apply.
 
Another thought here...are you absolutely certain the request is being made of list.smartfilter.com?  Is it possible that it is going to something like download1.list.smartfilter.com?  I'd suggest checking your logs to be sure, and just for testing you could open that domain name set to include *.smartfilter.com as well.





PaulLinton -> RE: SmartFilter Internet Database Download (23.Jan.2009 4:18:52 PM)

So I took your advice and dropped the additional users in the condition.  I also changed the list.smartfilter.com to just *.smartfilter.com as you suggested and applied the settings. 

Here are the options for the rule I am currently trying:
Name:       Allow InternetDatabase Download
Action:      Allow
Protocols:  All Outbound Traffic
From:        All Networks (and Local Host)
To:            Domain Name Set (System Policy Allowed Sites)
Condition: All Users

I started a query to watch what was going through the ISA server, then tried to pull the SmartFilter Internet Database.  It failed but I got the following info on the attempt:

Log type: Web Proxy (Forward)
Status: 12202 The ISA Server denied the specified Uniform Resource Locator (URL). 
Rule: 
Source: ( 192.168.1.251:0)
Destination: ( 216.38.163.83:80)
Request: POST /cgi-bin/updatelist
Filter information: Req ID: 093cad55 
Protocol: 
User: anonymous
Additional information
Client agent:
Object source: Processing time: 1
Cache info: 0x0 MIME type: 

I'm not sure why it is coming back as an anonymous user, but leaving that for now, from this info, I ran a ping to list.smartfilter.com.  It came back pinging "prpx.service.mirror-image.net" at the above mentioned IP.  I tried adding "*.mirror-image.net" to the System policy list and still nothing.  Then I tried adding it as an IP address range (216.38.163.83 to 216.38.163.84) with the same results.

Does this info help you?  Thanks again for all the speedy responses.




richardhicks -> RE: SmartFilter Internet Database Download (23.Jan.2009 4:43:08 PM)

Oh the fun of troubleshooting via e-mail (or in this case, forum posts!).  : )
 
Anonymous user is to be expected, since we didn't ask for any authentication on the access rule.  Also, since this is a web request, the ISA firewall doesn't do any reverse lookup in this case.
 
I'm sure we're missing something very simple here.  At this point, let's back up a bit and create a new access rule.  Back out the changes you made to the system policy and create a new access rule in the standard firewall policy.  The source will be the localhost, destination *.smartfilter.com, and the protocol will be HTTP.  Specify 'all users' only and lets test again.  If it fails, from the ISA firewall run the following command:
 
telnet list.smartfilter.com 80
 
Let me know what happens.  : )




PaulLinton -> RE: SmartFilter Internet Database Download (23.Jan.2009 6:35:14 PM)

Troubleshooting via email is difficult... I appreciate your patience with this problem so far. 
 
So, I created the new rule as suggested:
 
Name:       Starting Over
Action:      Allow
Protocols: HTTP
From:        Local Host
To:             URL Set (SmartFilter Database Site)
Condition: All Users
 
The URL Set includes both *.smartfilter.com and http://*.smartfilter.com.  Is URL set the only way to have it go to a site, or am I missing something?
 
Then I disabled the rule we were working with (as a precaution).  Fired up the SF Admin Console and tried grabbing the database... nothing.
 
As you advised, I opened a command window, typed in telnet list.smartfilter.com 80.  It attempted to connect.  I waited about 5 minutes for it before just closing the blank window.
 
Just for kicks, I thought to try the System Policy Allowed Sites under Domain Name Sets, just to see if changing from URL Set to that would work.  It didn't.
 
This looks right to me.  I'm not sure why it isn't working.  Any other suggestions?




richardhicks -> RE: SmartFilter Internet Database Download (23.Jan.2009 8:13:58 PM)

If you didn't immediately get a 'could not open connection to the host on port 80' reply, then you have connectivity and the rule is working.  That's good news at least!
 
I think I see the problem now though.  The Smartfilter download site itself is requires basic authentication.  When I read your initial post, I misinterpreted it to mean that you couldn't require basic authentication for your access rule.  My apologies. 
 
If the remote host is requesting authentication, you will have no choice but to supply it.  You'll do this by entering the user credentials in the Smartfilter software.  Don't specify a proxy, and don't specify credentials to use the proxy, however.  The access rule we've created doesn't require authentication. 
 
Since the remote host accepts only HTTP, those credentials will be passed in the clear.  There's nothing you can do on your side to protect that, unfortunately.
 
 




Jason Jones -> RE: SmartFilter Internet Database Download (24.Jan.2009 8:18:54 AM)

Hmmm...that's a secure product then [;)]




richardhicks -> RE: SmartFilter Internet Database Download (24.Jan.2009 10:54:14 AM)

Apparently!  Websense works a little different.  When the application downloads the URL database it connects to a CGI application.  You have to have a valid product key in order to download the latest update.  No authentication is required because the CGI app checks to make sure your product key is valid and if it is, you get the update.  If you simply browse to download.websense.com looking for it though, you'll get a static HTML page that redirects you to websense.com.




PaulLinton -> RE: SmartFilter Internet Database Download (26.Jan.2009 12:26:22 PM)

You were correct in your original understanding of the problem. The list.smartfilter.com site does not require authentication on our end.  They do not supply credentials for us to sign into their site - the admin console has these credentials hard-wired in, so when a link is established, the credentials are exchanged and we get the download. 

I did finally work through this problem over the weekend.  Here is more background and then the solution I came up with.

Our ISA server has a 192.168 address.  This makes it part of the Internal network... subject to the Internal network rules.  For SmartFilter to get the database through the internal network, if authentication is required, then it requires basic authentication (as relayed to me by their support team) and passes these credentials through our network before going out to the internet.

Under Configuration > Networks, Local Host uses the loopback address 127.0.0.1, so I opened it up to check the properties. "Enable Web Proxy clients" is active.  Authentication is set to Integrated and Basic, but "Require all users to authenticate" is off.

For a firewall rule, I modified the one we previously defined:
Name:  Starting Over
Action: Allow
Protocols: HTTP
From: ISA Server (127.0.0.1)
To: System Policy Allowed Sites (*.smartfilter.com)
Condition: All Users

On Smartfilter Admin Console, on the ISA plugin and under Set Advanced Options > Download Setup, I put the following:
Download Type: HTTP
Connection: list.smartfilter.com
Port: 80
Proxy Server: 127.0.01
Port: 8080
Proxy ID: {blank}
Proxy Password: {blank}

The rest I left as it was.  The reason I still put in proxy settings, is that it still needs to know where to go before heading out on the internet.  The ID and Password are left blank because our Local Host does not require authentication. 

I deployed the changes and now it works 100%.  The best part of this solution is that I'm not sending privileged account information as plain text over the net when trying to grab the database.

Thanks for all the advice and suggestions.  [:D]  Because of your help, I was able to think through this problem a bit more clearly.  I hope this thread helps other people as I'm sure this is not the first time (or the last) that someone ran into this.  Thanks again!




Jason Jones -> RE: SmartFilter Internet Database Download (26.Jan.2009 4:07:10 PM)

Nice...thanks for the update [:)]




FDannels -> RE: SmartFilter Internet Database Download (14.Nov.2009 11:30:46 AM)

Thanks for the thread it helped me solve the same problem with my network, but revealed new hurdles to over come. First was my connection being to slow, and intermittent, to download the sfcontrol file. After blocking all my users for the necessary bandwidth, I finally got the full 313mb control list. The problem now is as soon as it completed, the "Microsoft Windows Firewall" service stopped working, even after multiple restarts. When I remove the sfcontrol file from "C:\Program Files\Secure Computing\SmartFilter ISA Plugin", then restart, everything works. When I put the file back and restart, the service fails again.

Any thoughts or suggestions would be helpful and greatly appreciated.




Page: [1]