The site's firewall--Microsoft Internet Security and Acceleration Server (ISA), currently has three problems that we have opened up tickets with Microsoft to fix.
1) Status: 5/5/2006: RESOLVED by ISA SP2.
The first is that a filter crashes randomly (about once every 2 weeks) and causes the entire site to go down. (Update 3/16: Microsoft believes that the latest SP will fix this. It is being tested right now. Update 4/5: Still no reoccurence of this issue. If it doesn't reoccur this month then it will be considered resolved).
2) Status: 5/5/2006: RESOLVED by Microsoft workaround.
The second is that ISA reports that it cannot log certain information and shuts itself down...again causing the entire site to go down. This happens more frequently...1-2 times per week. (Update 3/16: Microsoft has given us a workaround of turning of MSDE, which appears to have problems under load. We are testing now to prevent this problem. Update 4/5: Still no reoccurence of this issue. If it doesn't reoccur this month then it will be considered resolved).
3) Status: MICROSOFT STILL WORKING ON THE PROBLEM: 4/5/2006: ISA stops serving web pages...alternating between "DNS server not found" and "out of memory". Sometimes http works while https doesn't and vice versa. Microsoft is working on the problem.