WWW10, May 1-5, 2001, Hong Kong.
ACM 1-58113-348-0/01/0005.
Another prominent story which 'DoS'-ed hundreds of machines is the Internet Worm [2][3][4]. But it was at the beginning of 2000 when a complete new quality of DoS attacks started to be used widely. So called Distributed Denial of Service attacks stroke a huge number of prominent web sites including Ebay, Amazon or Buy.com [5].
It is quite evident that as the attacking tools become more and more available in the cracking community (we intentionally avoid the term hacker here) the threats to services in the Internet become stronger. As the amount of monetary transactions that are handled over the Internet increases, this state cannot be accepted. Unfortunately we will see that there is no complete solution for this kind of attacks yet. The W3 Security FAQ [6] states that protection from attack tools like TFN, TFN2K or stacheldraht is an active field of current security research.
A possible classification of IT attacks according to the intention of the cracker could be [7]:
... an attack designed to render a computer or network incapable of providing normal services. The most common DoS attacks will target the computer's network bandwidth or connectivity. Bandwidth attacks flood the network with such a high volume of traffic, that all available network resources are consumed and legitimate user requests can not get through. Connectivity attacks flood a computer with such a high volume of connection requests, that all available operating system resources are consumed, and the computer can no longer process legitimate user requests.J.D. Howard defines DoS as [9]:
If computer hardware, software, and data are not kept available, productivity can be degraded, even if nothing has been damaged. Denial-of-service can be conceived to include both intentional and unintentional assaults on a system's availability. The most comprehensive perspective would be that regardless of the cause, if a service is supposed to be available and it is not, then service has been denied.The kind of DoS attacks that we are interested in is usually carried out via networks and one of the main targets of such attacks are high profile websites like the ones named in the introduction. As these sites usually have a lot of hardware at their disposal, they are much harder to attack than ordinary hosts connected to the Internet. The web sites are normally composed of several web-servers with a load balancing system in front and they have multi megabit network connections. Consequently attackers have discovered new ways of bringing these systems to its knees. They don't use single hosts for their attacks but they also cluster several dozens or even hundreds of computers to do a coordinated strike. The WWW Security FAQ [6] on Distributed Denial of Service (DDoS) attacks, as these form is called:
An attack, however, is an intentional act. A denial-of-service attack, therefore, is considered to take place only when access to a computer or network resource is intentionally blocked or degraded as a result of malicious action taken by another user. These attacks don't necessarily damage data directly, or permanently (although they could), but they intentionally compromise the availability of the resources.
A Distributed Denial of Service (DDoS) attack uses many computers to launch a coordinated DoS attack against one or more targets. Using client/server technology, the perpetrator is able to multiply the effectiveness of the Denial of Service significantly by harnessing the resources of multiple unwitting accomplice computers which serve as attack platforms. Typically a DDoS master program is installed on one computer using a stolen account. The master program, at a designated time, then communicates to any number of "agent" programs, installed on computers anywhere on the Internet. The agents, when they receive the command, initiate the attack. Using client/server technology, the master program can initiate hundreds or even thousands of agent programs within seconds.The aggregated bandwidth of this large number of agent programs is probably bigger than any website's uplink capacity. In fact one can even speculate on the effects of an attack to major IP routers or global exchange points which could render large parts of the Internet inaccessible. [10], [11] and [12] offer more information on the related technologies. We will also give some examples of common tools and attacks in one of the next sections.
First of all we could attack the clients themselves, which is kind of useless as a general DoS attack, because there are just too many of them. A first reasonable attack point for a cracker wanting to launch a DoS is the router that connects the site hosting the webserver to its ISP. This would effectively cut off all access to the website.
Another possible target is the firewall system, although firewalls usually tend to be (or at least should be) quite immune to direct attacks. On the other hand, the firewall is usually a bottleneck as all in- and outbound traffic needs to pass through it. An attack form with a very high load might stop the firewall from working properly. As an example: during our experiments some misconfigured SYN-flood attack overloaded our university's firewall and blocked most in- and outgoing traffic.
The load-balancer is another attack target. As more and more manufacturers of such systems will become aware of this threat there is a good chance that this system will be one of the main points of defending web sites from DoS attacks.
Next the individual webservers can be attacked. As there are usually multiple servers, the effort to overload them all is usually higher than to attack a single bottleneck like the load-balancer. On the other hand, most webservers use of-the-shelf hard- and software and are thus especially vulnerable to known DoS attacks.
Finally there may be supporting services like database servers etc. As foreign access to these servers should be blocked at the firewall, the chance of a direct DoS attack on one of this systems is fairly small. Nevertheless a tricky attack on the webserver may cause a flood of subsequent requests to e.g. a database system so that normal requests aren't processed anymore.
Some attacks may affect the hardware of a given system although such incidents are very rare. Theoretically, a network card or CPU could fail to work due to some data in a network packet it receives or processes. On the other hand, a simple overload of a 10 MBit/s Ethernet link is an attack based on the limitation of the hardware.
Attacks targeting the operating system or the TCP/IP stack of a host or router are more likely. Many attacks of this type are known, some are bugs that can be fixed, some are fundamental limitations of a protocol specification etc.
On the application level there are web servers, database servers, CGI scripts etc. These could be attacked as well.
Cisco 7xxx routers with IOS/700 Software Version 4.1(1) or 4.1(2) have a bug where a long password given at a telnet login session leads to a buffer overflow and a subsequent reboot. According to our criteria the attacked system is a router, the part of the system under attack is the operating system and it is a bug which has been fixed in subsequent releases. [13]
Jolt2 is an attack targeting most Microsoft Windows systems (Win 98, NT4 SP5 and SP6, Win 2000). Jolt sends a continuous stream of ICMP ECHO_REPLY fragments with specially tuned fragmentation parameters to the attacked host. The exact cause of the following action is not known as the code of Microsoft's Operating Systems isn't freely available. What can be observed is that the consumption of CPU and memory resources raises to 100% which renders the system is unusable. This may be used as an attack on webservers, load balancers or even firewalls, if they run one of the named operating systems. The attacks aim at the network stack of the operating system and the reaction is caused by a bug. Fixes are available from Microsoft. [14]
Microsoft's Internet Information Server versions 4.0 and 5.0 suffered from a so called server URL parsing bug. The decoding of escape sequences in URL strings was implemented very inefficiently. Submitting long strings with large amounts of escape characters effectively stopped the web server from working for a significant time. This attack is an attack against the webserver application. It is based on an implementation bug. A fix is available from Microsoft. [15]
Smurf attacks use so-called amplifier sites in order to multiply the amount of traffic that hits the destination. These attacks send ICMP_ECHO_REQUEST packets with a spoofed sender address to one or several subnet broadcast addresses. The packets are received and replied by as many stations as are connected to the subnet. The replies are sent directly to the spoofed address of the attack destination. Normally this leads to congestion in the target's local network connection. Often even the lines of the ISP to which the target is connected become overloaded. This attack class strikes all kinds of targets, no matter if routers or hosts. Most of the time it is used against webservers. The effects of the attack typically put a huge stress on the network and the system (both software and hardware) that has to parse and discard the incoming packets. It is not based on a bug, although many people today think that subnets that allow a broadcast ping are a faulty configuration [16][17][18].
The main principle of SYN flood attacks is to generate many half-open TCP connections by sending SYN packets to the target without replying to the following SYN-ACK packet. Most of the time the SYN packets also have spoofed source addresses of non-existent or currently inactive hosts. As today's hosts can typically handle only a limited number of half-open connections per port and discard new connection requests as long as the so called backlog queue is full, this effectively stops ordinary clients from connecting to the (web-) server. This attack form falls in the category of attacks to webservers or load-balancers which have a weakness in their TCP implementations. Part of the problem is the unlucky connection establishment procedure of the TCP protocol so it is not totally clear, weather it is an implementation bug or design flaw [19] [20] [21].
There had been a number of attacks on apache webservers like Apache MIME flooding [22] or Apache Sioux Attack [23]. With specially formatted HTTP requests it was possible to make the webserver use up huge amounts of memory. As soon as the system started swapping out server processes or used up the whole system memory, the server was effectively dead. These attacks both target a specific webserver software on the webserver systems. They exploit implementation bugs.
This concludes our list of DoS examples. Of course this is only a small selection, online archives [24] provide huge lists of bugs and exploits in former and current software versions. Many of these exploits were packaged into attack tools and some of them are used by DDoS tools in order to attack a site from many locations at once.
The architecture of these tools is very similar and in fact some tools are modifications of others. The actual attack is carried out by so called daemons. A number of daemons is controlled by a handler and finally these handlers are activated by the attacker using client tools. Figure 2 displays the architecture of such a system.
Even more frightening might be a scenario where the tools don't stop at DDoS attacks but automate the process of intruding hosts with daemons which again spread themselves to other hosts. This way automated hacking into hundreds of thousands of computers might be possible.
The firewall rules should include some sanity checks for source and destination addresses: Packets arriving from the Internet must not have a source address originating from the interior net, and vice versa. By rejecting packets from the interior net with a non-local source address, packet spoofing becomes impossible. This technique is known as ingress and egress filtering [31]. Even if a host is invaded by a hacker, these rules make it impossible to use that host as a platform for further attacks requiring spoofed packets (e. g. SYN-Floods [20], Smurf-Attacks [15][17], etc.).
As explained in chapter 2, there are several well-known kinds of DoS attacks that exploit implementation bugs and for many of them there are known countermeasures. Of course, it is highly recommended to take these measures by installing the appropriate security patches, enabling SYN cookies [32], etc.
In contrast to attacks focusing on implementation or protocol errors, it is rather difficult to defend against DoS or DDoS attacks which overload the systems network connection or local resources. These attacks usually put a heavy load on the target by making regular requests very rapidly. It is hard to distinguish if a web server is stormed by thousands of clients, or if there is a DoS attack in progress.
A simple way to force the problem of heavy load is to use a server farm together with a load balancer. This will help against small attacks, but not against a DDoS started from several hundred hosts. Furthermore, increasing the number of servers is rather expensive.
Many experts think that the only durable solution to this problem is to globally improve the security on all hosts in the Internet to take attackers the possibility to use other hosts for running daemons. This includes securing the local operating system, applying ingress- and egress filters and protecting sites with firewall systems [12][33][34].
We don't think that the global state of Internet security will become any better in the foreseeable future. While many companies will start improving their security, taking measures like the ones detailed above, the growth of the Internet will simply absorb these effects. Many new companies and even more individuals with permanent connections will storm the net and many of them won't have the time, budget or will to secure their systems. Some people think about incentives to force people to care for security. One example of such an action is the abolishment of flatrates and the network wide use of usage-based fees [33]. We don't think that users and industry will follow these propositions, so solutions to protect the Internet sites under attack need to be found. There are already a small number of propositions how to deal with certain attack forms.
A simple response to SYN floods is the "first packet reject" method: The first packet sent by every host is discarded. This renders the common SYN flood attack ineffective, but all normal connection requests are delayed to the first retransmit of the SYN packet. Thus, all requests will be considerably slower. On the other hand, the SYN flood attack may be simply adapted by sending each SYN packet twice.
Another technique is the so called moving target defense. Here the host under attack changes its IP address to avoid being attacked. The problem here is that the legitimate users of the system need to be informed that the IP address has changed which usually is done by updating the DNS system. Due to caching it can take up to a number of days until all clients are informed of the update. This is clearly unacceptable as the effects are as severe as any DoS attack can be. Furthermore attackers only need to incorporate DNS lookups into their tools in order to evade this protection.
We developed a solution that tries to enhance the protection from DoS and DDoS attacks to webservers. It uses a combination of publically available protection tools and augments them with a load monitoring tool that stops clients (or attackers) from consuming too much bandwidth.
Network address translation (NAT) transcripts every incoming packet and changes the destination IP from the load balancer's to the web server's IP. All outgoing traffic is transcripted alike. As all in- and outgoing traffic has to pass the load balancer, this is not an ideal solution for our purposes, as the load balancer may easily become a bottle neck.
Direct Routing Request Dispatching (DR) changes the MAC addresses of incoming packets to the MAC address of the web server and forwards the packets. Web servers may answer directly, bypassing the load balancer. All Webservers must reside in one IP subnet for this to work.
IP Tunneling (IPIP) is a solution where incoming packets are wrapped in an IPIP encapsulation and are tunneled to the webserver. At the tunnel end packets get unwrapped and are delivered to the webserver application.
We have chosen to use IPIP mode as this has no restrictions on subnets and showed to work very efficiently.
In our system there is a configurable number of input queues on the load balancer and output queues on the webservers. There is a default input queue which can consume as much bandwidth as available. If the Traffic Monitor in the load balancer detects a possible DoS attack it gradually slows down all incoming traffic from the origination IP address by assigning it to more and more slower queues with e.g. 1000 kBit/s, 600 kBit/s, 300 kBit/s and finally to a queue with only 100 kBit/s. If even this does not stop the attack, the IP address is blocked in a firewall list for a configurable amount of time. At the same time it directs the webservers to slow down the outgoing traffic to the attacker's IP address gradually.
The Traffic Monitor consists of a manager and a monitor program. The monitor is running on the load balancer and all webservers. It is implemented as 3 separate threads. Thread 1 monitors the network for packets destined to or originating from the virtual web servers address. The source IP address, the length and the time of occurrence are noted in a hashtable (with the IP address as key). Thread 2 checks the hash table every 3 seconds. If it finds that a certain IP address is emitting or receiving too many traffic or if the packet/size ratio falls under a certain amount it marks the IP address as a potential attacker. Thread 3 is a server thread which listens for commands from the manager. The manager polls all the monitors in regular intervals for a list of potential attackers. It analyzes the supplied data and decides whether a potential attacker is indeed categorized as malicious. In this case the manager instructs the monitors to downgrade the attackers IP address to a lower queue or block it at all. After a certain interval of normal activity, IPs can be upgraded to better queues.
The manager sorts the IPs in one of several classes based on the data it receives from the monitors. Class 1 IPs have produced too much traffic with very small packets. As this is very likely a DoS attack, new packets from that source are totally blocked at the ipchains filter. Class 2 IPs use too much bandwidth over a long time. They get downgraded into a lower queue. Class 3 IPs use too much bandwidth but only for a short time. This may as well be a peak from an ordinary client. These IPs are now under suspicion. A timer is started and if the behavior of this IP persists it is put into class 2. Finally there is class 4. IPs in this class don't produce or consume much bandwidth but they send lots of very small packets. Again there is a timer and if this behavior is shown for a certain amount of time, a ipchains filter blocks traffic from that IP.
All filters and queues have associated expiration timers. After expiration the filter is deleted and the queue is upgraded to the next higher class.
Normal attacks like TCP SYN floods did not show any significant degradation of system performance but where handled effectively by the Linux SYN Cookie mechanism. UDP floods were blocked by the ipchains filters. Of course it is still possible to overload the network connection at some point in front of our system. The normal consequence would be to install the filters as early in the traffic flow as possible blocking everything except HTTP traffic (tcp/80).
We use the following tools for attacks. http_load written by Jef Poskanzer [37] which runs multiple HTTP fetches in parallel to test the throughput of a web server. It runs as a single process so it doesn't take to much CPU time. In our test the program reads 100 URLs from a file and tries to fetch them from the webserver randomly over a specified period of time. http_load uses 64 threads in parallel which try to fetch as many pages as possible in 210 seconds of time. We use three different URL sets: set one consists of 100 static html pages, set two consists of 100 URLs which access a CGI script with different parameters. Finally set 3 consists of 34 URLs accessing the CGI script and 66 static html pages.
The other test tools are TFN2K (see section 2.4.3) and SYN Flooder from a hacker called Zakath. SYN Flooder performs a heavy SYN-flood attack. We slightly modified it so we can do SYN-flood attacks with randomly generated IP addresses.
For simulating an ordinary web client, we decided to run http_load with a single thread and the mixed URL database.
As a reference we ran this tool several times without any attack to the web site. Then we ran the tool while the system was under attack with and without the traffic shaping monitor tool active. In a first run, all attack hosts use the same attack at a time. We tested the following cases
The sniffer is inserted at 3 positions:
BP1 (break point 1): Between the backbone switch and the load
balancer. Here all incoming traffic can be gathered. This is important
because not all traffic is passed to the webservers.
BP2 (break point 2): Between the load balancer and the workgroup
switch. Here all data can be gathered that is forwarded to the webservers.
This way we can determine how much traffic has already been blocked by
the load balancer.
BP3 (break point 3): Between the backbone and the workgroup
switch. Here all outgoing traffic can be measured. This is all the traffic
produced by the webservers.
Due to limited space, we will present here the results of only two of the tests, namely the http-attacks with a static and a CGI database. These tests show how our traffic monitor reacts to http overload attacks under different conditions.
ArrowPoint [38] is producing switches with load-balancing capabilities. These switches have been extended to provide some security against DoS attacks. Fragmented packets or packets being too short are dropped. Further, some checks on the IP addresses are done (e. g. source and destination address must be different, loopback or multicast addresses are not allowed). TCP connections are parsed, too. The 3-way handshake must be completed within 16 seconds, otherwise these packets are dropped. Another useful feature is the possibility to block packets directed to special ports, or originating from certain IP address ranges. Network address translation (NAT) is provided, too.
F5 is the manufacturer of BigIP, a load balancer switch with several security-relevant features. First, BigIP provides a monitor tool to watch the network traffic. Attack attempts will be noted in a log file. BigIP provides port mapping (i.e. packets for a special port are transparently redirected to a different host) and NAT. Further, packet filtering is provided. Packets belonging to several kinds of attack (e.g. Teardrop, Land, Ping of Death) are recognized and will be discarded [39].
Both manufacturers provide firewall load balancing: The traffic is equally shared on several firewalls; the switch makes sure that all packets belonging to the same stream will be routed through the same firewall. This allows both load balancing and the possibility to set up a redundant backup firewall.
[2] E.H. Spafford, The Internet Worm Program: An Analysis. Purdue Technical Report CSD-TR-823, Department of Computer Sciences Purdue University, West Lafayette, IN. 1988.
[3] D. Seeley, A Tour of the Worm. Department of Computer Science, University of Utah. 1988.
[4] Eichin, M. and Rochlis, J. With Microscope and Tweezers: An Analysis of the Internet Virus of November 1988. Massachusetts Institute of Technology, 1988.
[5] M. Williams, Ebay, Amazon, Buy.com hit by attacks, IDG News Service, 02/09/00, http://www.nwfusion.com/news/2000/0209attack.html - visited 18.10.2000.
[6] L. Stein, The World Wide Web Security FAQ, Version 2.0.1, http://www.w3.org/Security/Faq/ - visited 04.10.2000.
[7] W.R. Cheswick, S.M. Bellovin, Firewalls and Internet Security, Addison Wesley Longman 1994
[8] Attrition Mirrored Sites, http://Attrition.org/mirror/attrition/ - visited 03.11.2000.
[9] Dr. J.D. Howard, An Analysis of Security Incidents on the Internet 1989 - 1995, Carnegie Mellon University, Carnegie Institute of Technology, http://www.cert.org/research/JHThesis/ - visited 02.11.2000.
[10] J. Elliot, Distributed Denial of Service Attacks and the Zombie Ant Effect, IT Professional, Mar./Apr. 2000, pp55-57.
[11] K.T. Fithen, Internet Denial of Service Attacks and the Federal Response, Testimony before the Subcommittee on Crime of the House Committee on the Judiciary and the Subcommittee on Criminal Justice Oversight of the Senate Committee on the Judiciary, February 29, 2000, http://www.cert.org/congressional_testimony/Fithen_testimony_Feb29.html - visited 10.11.2000.
[12] Results of the Distributed-Systems Intruder Tools Workshop, Pittsburgh, Pensilvania USA, November 2-4 1999, CERT Coordination Center, Software Engineering Institute, Carnegie Mellon University, Pittsburgh, http://www.cert.org/reports/dsit_workshop.pdf - visited 12.11.2000
[13] Field Notice: 7xx Router Password Buffer Overflow; Revision 1: December 15 1997; http://www.cisco.com/warp/public/770/pwbuf-pub.shtml - visited 18.10.2000.
[14] Microsoft Security Bulletin (MS00-029): Patch available for "IP Fragment Reassembly" Vulnerability; May 19, 2000; http://www.microsoft.com/technet/security/bulletin/ms00-029.asp - visited 18.10.2000.
[15] Microsoft Security Bulletin (MS00-23): Patch available for "Myriad Escaped Characters" Vulnerability; April 12, 2000; http://www.microsoft.com/technet/security/bulletin/ms00-023.asp - visited 18.10.2000.
[16] K. Wooding, Magnification Attacks - Smurf, Fraggle, and Others; http://www.codetalker.com/whitepapers/dos-smurf.html - visited 19.10.2000
[17] C.A. Huegen, The Latest in Denial of Service Attacks: "Smurfing" Description and Information to Minimize Effects; http://www.pentics.net/denial-of-service/white-papers/smurf.cgi - visited 19.10.2000
[18] CERT Advisory CA-98.01 "smurf" IP Denial-of-Service-Attacks, January 5, 1998, http://www.cert.org/advisories/CA-1998-01.html - visited 23.10.2000
[19] daemon9, route infinity, TCP SYN Flooding Attacks; Phrack magazine, Vol. 7, Issue 48, File 13 of 18, July 1996
[20] C.L.Schuba, I.V. Krsul, M.G. Kuhn, E.H. Spafford, A. Sundaram, D. Zamboni, Analysis of a Denial of Service Attack on TCP, Coast Laboratory, Department of Computer Science, Purdue University
[21] CERT Advisory CA-96.21, September 19, 1996, TCP SYN Flooding and IP Spoofing Attacks, http://www.cert.org/advisories/CA-1996-21.html - visited 23.10.2000
[22] Web servers / possible DOS Attack / mime header flooding (archive), http://www.securityfocus.com/archive/1/{10516|10520|10521|10525|10526} - visited 23.10.2000
[23] YA Apache DoS attack (archive), http://www.securityfocus.com/archive/1/10228 - visited 23.10.2000
[24] Rootshell.com, http://www.rootshell.com/ - visited 08.02.2001
[25] D. Dittrich, The DoS Project's "trinoo" distributed denial of service attack tool, October 21, 1999, http://staff.washington.edu/dittrich/misc/trinoo.analysis.txt - visited 13.11.2000
[26] Phrack Magazine, Volume Seven, Issue Forty-Nine, File 06 of 16, [ Project Loki ], http://www.phrack.com/search.phtml?view&article=p49-6
[27]Phrack Magazine Volume 7, Issue 51 September 01, 1997, article 06 of 17 [ L O K I 2 (the implementation) ] http://www.phrack.com/search.phtml?view&article=p51-6
[28] D. Dittrich, The "Tribe Flood Network" distributed denial of service attack tool, October 21, 1999, http://staff.washington.edu/dittrich/misc/tfn.analysis.txt - visited 13.11.2000
[29] J. Barlow, W. Thrower, TFN2K - An Analysis, AXENT Security Team, February 10, 2000 (Updated March 7, 2000) Revision: 1.3, http://packetstorm.securify.com/distributed/TFN2k_Analysis-1.3.txt - visited 13.11.2000
[30] David Dittrich, The "stacheldraht" distributed denial of service attack tool, December 31, 1999, http://staff.washington.edu/dittrich/misc/tfn.analysis.txt - visited 13.11.2000
[31] P. Ferguson, D. Senie, RFC 2267, Network Ingress Filtering: Defeating Denial of Service Attacks which employ IP Source Address Spoofing, Cisco Systems Inc., BlazeNet Inc., January 1998
[32] D.J. Bernstein, SYN Cookies, ftp://koobera.math.uic.edu/syncookies.html - visited 13.11.2000
[33] X. Geng, A.B. Whinston, Defeating Distributed Denial of Service Attacks, IEEE IT-Pro, July | August 2000
[34] Submissions to the Paketstorm DDOS paper constest, http://packetstorm.securify.com/papers/contest/ - visited 13.11.2000
[35] Linux Virtual Server - http://www.linuxvirtualserver.org/
[36] Linux Advanced Routing HOWTO
[37] http_load by Jef Poskanzer, from http://www.acme.com/software/ - visited 10.02.2001
[38] Arrowpoint, Whitepaper: Web Site Security and Denial of Service Protection, http://www.arrowpoint.com/solutions/white_papers/printer/Web_Site_Security.html - visited 12.11.2000
[39] F5: Whitepaper: A Defense To Denial of Service Attacks and Other Cyber Threats, http://secure.f5.com/solutions/whitepapers/defense.html - visited 12.11.2000