Microsoft.com: What's the story?
If you've ever wondered how microsoft.com uses our technology then read on. I recently came across some good information from the folks over at the Operations team at Microsoft.com. The thread basically talks about how we use IIS, Firewalls and Windows Server 2008. I think as we come up to launch next year it's a really good and quick insight into what they do and how they do it. So enjoy the reading and let me know what you think..Pretend I've asked about how they protect our sites...
At this point we still don't use firewalls for MS.COM sites and don't have any plans on the books to put them in place. Here is the short answer as to why:
We don't handle HBI data so we don't have the need for external logging capabilities. If we did handle HBI, we'd have firewalls.
We have ~650GB/day of IIS logs just for www.microsoft.com and update.microsoft.com (not including the 6GB/hour for each download server). Just IIS logs are a challenge without trying to parse another ~650GB of firewall logs.
5+ years ago, there wasn't a firewall solution that would scale to our needs and this forced us to focus on network, host, and application security. Based on the success of that work, we've not looked further at firewalls even though there are solutions that I believe (haven't tested) would handled the traffic load (our non-download based web traffic alone can be in the 8-9 Gbps range and ~30 total for internal hosted traffic).
We also used NLB for load balancing exclusively up until July 2006 and the micro segmentation of networks required by that solution made firewalls an expensive and very complex solution. Again, especially at the scalability that used to be available.
Application security is critical since a firewall is likely going to allow traffic on the correct port and protocol through to the web servers so IIS/ASP.NET/Applications must deal with these requests gracefully. I realize there are other options/features of firewalls/IPS that provide other options.
In terms of how we protect the sites, we utilize (starting at the outside edge of the network and working in):
Cisco Guards for DoS detection and automated response
Router ACLs are in place to block unnecessary ports
NetScalers for www.microsoft.com and MSDN/TechNet (NLB still for update.microsoft.com) and those also provide DoS protection inherently as well as providing a few other knobs we can turn when required.
Windows and IIS...rock solid and secure! www.microsoft.com is on Windows Server 2008/IIS7, MSDN/TechNet are migrating to Win2k8/IIS7, and update.microsoft.com is on Windows Server 2003/IIS6. We do all the normal shut-off-unused-services practices that line up with MS published security guidance and we utilize GFS images to ensure standardized builds of systems.
Automated Netmon/Perfmon captures for attack analysis on NLB systems when SYN floods occur (event trigger). We've not yet done this for NetScaler systems, but we are noodling on how in our copious spare time :).
We do run AV on our servers when we can. At times product adoption means we don't install it, but we do normally run AV.
Application security as mentioned. ACE is very good resource for this aspect. ACE is an internal team that does threat modelling for applications.
So there you have it. I think this is a good insight into how we run our own internet properties today. What do you think? Have you got any feedback for the boys over at our MSCOM Operations team?
