• RSS
  • Twitter
  • FaceBook

Welcome to ISAserver.org

Forums | Register | Login | My Profile | Inbox | RSS RSS icon | My Subscription | My Forums | Address Book | Member List | Search | FAQ | Ticket List | Log Out

Microsoft Firewall service stopping

Users viewing this topic: none

Logged in as: Guest
  Printable Version
All Forums >> [ISA Server 2004 Cache] >> General >> Microsoft Firewall service stopping Page: [1]
Login
Message << Older Topic   Newer Topic >>
Microsoft Firewall service stopping - 16.Aug.2010 9:24:15 PM   
josefah

 

Posts: 10
Joined: 16.Aug.2010
Status: offline
-Please Help!!

     I have having a problem with Microsoft Firewall service periodically stopping on my Enterprise network.
     I am getting the event errors (Disk cache E:\urlcache\Dir1.cdat failed to initialize. Some errors were encountered when ISA Server restored specific data cache files. ISA Server will now attempt to recover these files. These errors may have occurred because there was not enough time to complete all necessary shutdown operations, when ISA Server was previously shut down. To avoid this in future, you can increase the value of the HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\WaitToKillServiceTimeout registry key), (The total Firewall log size has reached 80%% of the total log file size limit.),(The ISA Server Web filter failed to log information to file ISALOG_20100816_WEB_000.w3c in path E:\ISALogs\Web Proxy. The data is the error code.) and (The Firewall service stopped because an application filter module C:\Program Files\Microsoft ISA Server\w3filter.dll generated an exception code C0000005 in address 64742375 when function CompleteAsyncIO was called. To resolve this error, remove recently installed application filters and restart the service.)
    I increased this registry value from 30000 to 300000. I gave everyone full permissions to the "E:\urlcache\Dir1.cdat " file.  I don't know how to change the size of the log file. I Also made the ISALOG an antivirus exception.
    I've been working with computers for a long time but I'm new with servers to please be as simple as possible.
   Thank you very much for any assistance

Very Respectfully,
Josefah

< Message edited by josefah -- 16.Aug.2010 10:28:48 PM >
Post #: 1
RE: Microsoft Firewall service stopping - 17.Aug.2010 3:50:32 PM   
Rotorblade

 

Posts: 1348
Joined: 27.Feb.2007
Status: offline
quote:


The ISA Server Web filter failed to log information to file ISALOG_20100816_WEB_000.w3c in path E:\ISALogs\Web Proxy. The data is the error code.) and (The Firewall service stopped because an application filter module C:\Program Files\Microsoft ISA Server\w3filter.dll generated an exception code C0000005 in address 64742375 when function CompleteAsyncIO was called. To resolve this error, remove recently installed application filters and restart the service.)



The Firewall service will stop if it cannot log. Since you're using flat file logging; you can increase the log file size by modifying the logging setting options (Firewall and Web) under logging tasks on the Montoring pane in the ISA MMC.

hth

RB

_____________________________

David Melvin
Ohio
MCSE: Security 2003, MCSA:Security 2003

(in reply to josefah)
Post #: 2
RE: Microsoft Firewall service stopping - 17.Aug.2010 6:03:22 PM   
josefah

 

Posts: 10
Joined: 16.Aug.2010
Status: offline
Thank you.  Did this, i will see if the issue continues during our maintenance hours. 2-4 am. 

_____________________________

Very Respectfully,
Josefah

(in reply to Rotorblade)
Post #: 3
RE: Microsoft Firewall service stopping - 17.Aug.2010 8:23:12 PM   
josefah

 

Posts: 10
Joined: 16.Aug.2010
Status: offline
  -Rotorblade  

       I made the changes you suggested no luck. It delayed the problem about 30 minutes but the exact same errors.
       I did notice another problem with the URL Cache though the file the E:\urlcache\dir1.cdat file is 25 gigs. the "max sized of URL cached in memory" is 12800 bytes. I think this may be some sort of problem.
       If I stop the firewall delete this file then restart the firewall, will it rebuild itself?
       Your help is greatly appreciated.

Very Respectfully
Josefah
ET3      US Navy

(in reply to josefah)
Post #: 4
RE: Microsoft Firewall service stopping - 18.Aug.2010 5:08:29 PM   
Rotorblade

 

Posts: 1348
Joined: 27.Feb.2007
Status: offline
quote:


I did notice another problem with the URL Cache though the file the E:\urlcache\dir1.cdat file is 25 gigs. the "max sized of URL cached in memory" is 12800 bytes. I think this may be some sort of problem.
      If I stop the firewall delete this file then restart the firewall, will it rebuild itself?


No, you don't want to be doing that! Sounds like you might have two problems going? Are you sure you're not having some I/O issues? How is the disk space? Is caching configured on the same disk volume as the ISA log files?

To properly clear the cache; you need to disable it, then enable and set new. It may be an issue that the cache is corrupt?


On the Logging issues; check and make sure you have plenty of disk space (and the Caching is not on same volume!) and you have properly configured your flat file logging option maintain storage limits to delete as necessary. Make sure compression is on? It sounds like a heck of a lot of logging is taking place? What is showing on the ISA monitor and in the logs?

Go NAVY!


_____________________________

David Melvin
Ohio
MCSE: Security 2003, MCSA:Security 2003

(in reply to josefah)
Post #: 5
RE: Microsoft Firewall service stopping - 18.Aug.2010 5:25:58 PM   
Rotorblade

 

Posts: 1348
Joined: 27.Feb.2007
Status: offline
quote:


(The Firewall service stopped because an application filter module C:\Program Files\Microsoft ISA Server\w3filter.dll generated an exception code C0000005 in address 64742375 when function CompleteAsyncIO was called. To resolve this error, remove recently installed application filters and restart the service.)



On the above error; what ISA version are you running? There are some known issues depending on what SP level you are at. It could also be that you have a third-party add-on filter that is giving you grief?

_____________________________

David Melvin
Ohio
MCSE: Security 2003, MCSA:Security 2003

(in reply to josefah)
Post #: 6
RE: Microsoft Firewall service stopping - 18.Aug.2010 5:40:36 PM   
Rotorblade

 

Posts: 1348
Joined: 27.Feb.2007
Status: offline
quote:


I Also made the ISALOG an antivirus exception.


I just caught this too;- What else is running on this server? Is this a SMB server? Installing Antivirus engines other than AV engines specifically designed to work as an add-on filter to ISA is not recommended and not necessary. It’s a firewall and not a workstation. Host based AV engines can cause issues with ISA and should not be installed.  


_____________________________

David Melvin
Ohio
MCSE: Security 2003, MCSA:Security 2003

(in reply to josefah)
Post #: 7
RE: Microsoft Firewall service stopping - 18.Aug.2010 7:41:39 PM   
josefah

 

Posts: 10
Joined: 16.Aug.2010
Status: offline
    Well it turns out that i didn't make the Firewall and Web Logging setting big enough. I gave them more than enough room now so that should be good.

   This is a secondary domain controller and ISA server.  We are running Micrsoft ISA 2004 and symantec antivirus.

   To get the AV uninstalled would take an act of God. I can try to explain that it does nothing but mess things up.


Very Respectfully,
Josefah
ET3      US Navy

(in reply to Rotorblade)
Post #: 8
RE: Microsoft Firewall service stopping - 19.Aug.2010 12:01:57 PM   
josefah

 

Posts: 10
Joined: 16.Aug.2010
Status: offline
The problem is good now we went the entire day with no errors. It was the Firewall and Web Logging like you said.

However i have a question. To prevent this problem in the future, is there a way to automatically task the logs to automatically delete after a certain amount of time?

Thank You very much for help.

Very Respectfully,
Josefah
ET3      US Navy

(in reply to josefah)
Post #: 9
RE: Microsoft Firewall service stopping - 19.Aug.2010 3:58:24 PM   
Rotorblade

 

Posts: 1348
Joined: 27.Feb.2007
Status: offline
Glad to hear that the issued is resolved for now and I fully understand where you’re coming from in respect to taking an act of god to get things changed. I suspect this is a Gov/Mil installation and it’s obvious that some of your superiors skip over the best practice section of the ISA installation manual!

  As for your question: Yes, there are logging option settings that specify log storage limits; just set them as necessary. I would recommend to you that SQL logging would be the better option but as you said, it would probably take a act of god to get that changed too!

Best regards,

RB  

_____________________________

David Melvin
Ohio
MCSE: Security 2003, MCSA:Security 2003

(in reply to josefah)
Post #: 10
RE: Microsoft Firewall service stopping - 19.Aug.2010 4:49:37 PM   
josefah

 

Posts: 10
Joined: 16.Aug.2010
Status: offline
Yes i might be able to get that SQL logging implemented. It ill talk to some people or just try and sneak it in.

I really do appreciate your help.  My network is for 5000 plus people and im new at the job.  I'm kinda ignorant

Very Respectfully,
Josefah
ET3      US Navy

< Message edited by josefah -- 19.Aug.2010 9:27:44 PM >

(in reply to Rotorblade)
Post #: 11
RE: Microsoft Firewall service stopping - 19.Aug.2010 9:21:11 PM   
josefah

 

Posts: 10
Joined: 16.Aug.2010
Status: offline
Well I spoke too soon. 48 hours with no problem the same problem as before.

The first error is "The ISA Server Web filter failed to log information to file ISALOG_20100817_WEB_000.w3c in path E:\ISALogs\Web Proxy. The data is the error code."  Which I've seen i increase the size of the max log size.

Following that is "ISA Server failed to write content to the cache file." about ten times.

Then the error "The Firewall service stopped because an application filter module C:\WINNT\system32\msvcrt.dll generated an exception code C0000005 in address 77BC6F76 when function CompleteAsyncConnect was called. To resolve this error, remove recently installed application filters and restart the service."  and "Faulting application wspsrv.exe, version 4.0.2167.909, stamp 4977a087, faulting module msvcrt.dll, version 7.0.3790.3959, stamp 45d70b06, debug? 0, fault address 0x00026f76." and "The Firewall service stopped because an application filter module C:\Program Files\Microsoft ISA Server\w3filter.dll generated an exception code C0000005 in address 64742375 when function CompleteAsyncIO was called. To resolve this error, remove recently installed application filters and restart the service." then some more cache errors.

Unless more than one application filter is causing then problem. I just don't think it's likely to have a similar error from two application filters.

I still think that its a problem with logging because that's the error that appears to be setting everything off. I talked to a more senior tech than myself. He thinks it may be a cache problem. 

Any ideas??  im totally lost. Ill continue to research but im kinda stuck.

thank you

Very Respectfully,
Josefah
ET3      US Navy

(in reply to josefah)
Post #: 12
RE: Microsoft Firewall service stopping - 20.Aug.2010 4:42:23 PM   
Rotorblade

 

Posts: 1348
Joined: 27.Feb.2007
Status: offline
Well, you're going to have to step through this one piece at a time because I believe you have a couple of things going. I first would disable logging for a while to see if ISA stays running. Also, take a look at monitoring and the logs to see what is being logged. It sounds like you have something really going to town on your ISA. Secondly, what version of ISA are you currently running? There are a couple of known issues with that particular fault. If that doesn’t solve the riddle then I would be looking very hard at the AV that is installed and not to mention that you’re also have this as a DC which is another No-No! It’s going to have to be a process of elimination. Stepping through and disabling each filter one-at-a-time is another way to troubleshoot. It doesn’t sound like a cache problem unless you’re having I/O issues. I would probably lay odds on the logging or AV engine. You can also try to trap the faulting culprit using the Microsoft’s IIS exception monitor and debugging tool. I have used this tool many times to find “needles in the haystack”

RB

http://www.microsoft.com/downloads/details.aspx?FamilyID=28bd5941-c458-46f1-b24d-f60151d875a3&DisplayLang=en    

_____________________________

David Melvin
Ohio
MCSE: Security 2003, MCSA:Security 2003

(in reply to josefah)
Post #: 13
RE: Microsoft Firewall service stopping - 20.Aug.2010 5:46:16 PM   
josefah

 

Posts: 10
Joined: 16.Aug.2010
Status: offline
Thanks for the tool.

I know AV's and ISA are not supposed to be on DC's but i can't really change that. I disabled web proxy logging. 

We are running Microsoft ISA server 2004. I set the firewall logs to automatically delete after 5 days but they continue to grow. (From 13.5 gigs 3days ago to 17.5 gigs now.) 

I'm gonna sound dumb but how exactly do you look at the log in monitoring? Click query? 

thank you

Very Respectfully,
Josefah
ET3      US Navy

(in reply to Rotorblade)
Post #: 14
RE: Microsoft Firewall service stopping - 23.Aug.2010 8:44:54 AM   
Rotorblade

 

Posts: 1348
Joined: 27.Feb.2007
Status: offline
quote:


We are running Microsoft ISA server 2004.


If you can provide the specific version # that would help; go to the ISA MMC, click General - Click view ISA server computer details.

quote:

 
I'm gonna sound dumb but how exactly do you look at the log in monitoring? Click query? 


Afirm; you can edit and define the query details. Using monitoring will help you a lot in situations as yours.

The other tool you may want to run is the ISA BPA! This is a great tool to help find things that you may have overlooked.

http://www.microsoft.com/downloads/details.aspx?familyid=D22EC2B9-4CD3-4BB6-91EC-0829E5F84063&displaylang=en


_____________________________

David Melvin
Ohio
MCSE: Security 2003, MCSA:Security 2003

(in reply to josefah)
Post #: 15
RE: Microsoft Firewall service stopping - 24.Aug.2010 11:20:20 PM   
josefah

 

Posts: 10
Joined: 16.Aug.2010
Status: offline
One of my co worker made a breakthrough discovery that kind of takes us in a whole new direction. 

Right before every single crash there are four information events: 1. Event ID 2001
wins (3196) Shadow copy 1 freeze started. 2. Event ID 2001
lsass (512) Shadow copy 1 freeze started. 3.Event ID 2003 lsass (512) Shadow copy 1 freeze stopped. 4.Event ID 2003 WINS (3196) Shadow copy 1 freeze started.
Then the normal errors and a firewall crash. 
 
I am currently researching this issue but any ideas will help. I can't believe i didn't notice this before.
 
thank you

Very Respectfully,
Josefah
ET3      US Navy


 

(in reply to Rotorblade)
Post #: 16
RE: Microsoft Firewall service stopping - 25.Aug.2010 9:18:24 AM   
Rotorblade

 

Posts: 1348
Joined: 27.Feb.2007
Status: offline
Ok,that helps. Are you performing backups on this server? If so, what product are you using and are there any backup agents installed? As I have previously stated before; ISA is a Firewall and security appliance and you need to treat it as such. You wouldn’t be running AV and other applications on a Cisco ASA appliance would you? Performing backups on ISA is probably interfering with the I/O process and is going to cause issues.

RB

_____________________________

David Melvin
Ohio
MCSE: Security 2003, MCSA:Security 2003

(in reply to josefah)
Post #: 17
RE: Microsoft Firewall service stopping - 25.Aug.2010 1:26:55 PM   
Rotorblade

 

Posts: 1348
Joined: 27.Feb.2007
Status: offline
Is this server a SBS instance? I also ran across this KB - http://support.microsoft.com/kb/830575  which could be related to your issue. Since you’re using flat file logging, was MSDE uninstalled recently? If so then possibly this could be the issue with VSS. I would stop doing backups, if you are and see if the issues resolve.    

_____________________________

David Melvin
Ohio
MCSE: Security 2003, MCSA:Security 2003

(in reply to josefah)
Post #: 18
RE: Microsoft Firewall service stopping - 25.Aug.2010 1:35:22 PM   
Rotorblade

 

Posts: 1348
Joined: 27.Feb.2007
Status: offline
Also... check to see if VSS is enabled on the ISA log volume and Cache volume?

_____________________________

David Melvin
Ohio
MCSE: Security 2003, MCSA:Security 2003

(in reply to Rotorblade)
Post #: 19
RE: Microsoft Firewall service stopping - 3.Sep.2010 9:49:10 AM   
mahaddon

 

Posts: 19
Joined: 3.Apr.2009
Status: offline
Not sure if this is helpful or not but we were experiencing firewall service crashes and event log entries relating to 3rd party filters too.  We were using Captivate to provide an internet usage agreement page.  We disabled and uninstalled Captivate and haven't had the problem since and that was several months ago.

(in reply to Rotorblade)
Post #: 20

Page:   [1] << Older Topic    Newer Topic >>
All Forums >> [ISA Server 2004 Cache] >> General >> Microsoft Firewall service stopping Page: [1]
Jump to:

New Messages No New Messages
Hot Topic w/ New Messages Hot Topic w/o New Messages
Locked w/ New Messages Locked w/o New Messages
 Post New Thread
 Reply to Message
 Post New Poll
 Submit Vote
 Delete My Own Post
 Delete My Own Thread
 Rate Posts