23 Sept. 2002 Denying access to probing attacks using the CHECK_404 plugin I. Introduction II. Installation II.a. Installing the initialization procedure II.b. Installing the pre-request procedure II.c. Installing the post-response procedure III. How does CHECK_404 work. IV. Specifying response options IV.a. Parameters in CHK_PRE.REX IV.b. Parameters in CHK_POST.REX V. The penalty addon ------------------ I. Introduction The CHECK_404 plugin for SREhttp/2 is used to deny access to clients who've committed too many "404" requests (requests for non-existent resources) in the past several minutes. The reason for doing this is to stop hackers who are randomly searching for system files. The basic idea is: If an excessive number of "404" (resource not found) responses have been recently sent to an ip address, then this client is assumed to be an attacker, and no further requests from him will be acceted.. This assumption is maintained for several minutes, after which the slate is wiped clean (the client is assumed to now be innocent, until the next set of suspicious requests). ------------------ II. Installation: The CHECK_404 plugin consists of 3 components: a) A SRE2003 initialization procedure b) A SRE2003 pre-request procedure c) A SREhttp/2 post-response procedure Installing these components is simple: all you have to do is make a few changes to parameters in SRE2003.CFG and in SREHTTP2.CFG. No files need to be copied or modified (SREhttp/2 installs all the needed files in the appropriate directories). *** Quick installation *** Run the SREhttp/2 configurator, and choose the Enable/Disable the Check-404 facility link. Make sure you restart SRE2003 -- a link is provided, in the response to "Enable/Disable the Check-404 facility", that will automatically restart SRE2003. *** Sort of quick installation *** i) Change INIT_PROCS, in SRE2003.CFG, to init_procs=srehttp2\sreh2ini.rxx, chk_init.rex ii) Change PRE_REQUEST, in SRE2003.CFG, to pre_request=chk_pre.rex iii) Change POST_RESPONSE, in PRELOADS.CFG, to post_response=procs\CHK_POST.REX After making these modifications, restart SRE2003 -- CHECK_404 will be on the job (using default parameters that in most cases need never be changed). Note that this installation is permanent -- every time you start SRE2003 & SREhttp/2, CHECK_404 is enabled. Of course, you can always uninstall CHECK_404 (by undoing steps i to iii). The next three sections describe steps i to iii in greater detail ... ------------------ IIa. Installing the initialization procedure Installing the initialization procedure is done by modifying the SRE2003 init_procs configuration parameter (in SRE2003.CFG). For example, the default SRE2003.CFG used by SREhttp/2 has: init_proc=srehttp2\SREH2INI.RXX You should change that to: init_proc=srehttp2\SREH2INI.RXX, chk_init.rex Note that CHK_INIT.REX should be in the SRE2003\BIN directory. You can also change the default values of several parameters when launching CHK_INIT.REX. To do this, just enter the parameters immediately after the CHK_INIT.REX The syntax is: INIT_PROC = 'CHK_INIT.REX max404 duration ' Where: max404 == threshold number of 404 responses (default=8) duration == duration (in minutes) this threshold is computed over (default=5). If more then max404 "404 responses" have been sent in the last "duration" minutes, deny access. Notes: * to specify duration you must also specify max404 * SRE2003.CFG should be in your SRE2003\CFG directory. * Note that the use of duration is liberal in an unsophisticated kind of way -- as soon as a client slips beneath the threshold, access is allowed. This means that a slow and steady stream of probes will not be thwarted. You can set a high value for duration to deal with this, but this may end up penalizing sloppy-but-honest clients. If this becomes a problem, let us know (danielh@crosslink.net) and a more sophisticated algorithim (say, one that credits non-404 responses) may be developed. Example: init_procs=chk_init.rex 5 10 , srehttp2\sreh2ini.rxx Examples (abstracting from other initialization procedures that might also be listed in INIT_PROCS): a) Use the default parameters: init_procs='CHK_INIT.REX ' b) Set max404=10, and use the default value of duration INIT_PROC ='CHK_INIT.REX 10 ' ------------------ IIb. Installing the pre-request procedure Installing the initialization procedure is done by modifying the SRE2003 pre_request configuration parameter (in SRE2003.CFG). For example, the default SRE2003.CFG used by SREhttp/2 has: pre_request=0 You should change that to: pre_request=chk_pre.rex Notes: * more then one pre-request procedure can be specified. For example, if you are using the simple load balancer, then you could use.. pre_request=balance1.rxx , chk_pre.rex * CHK_PRE.REX should be in the SRE2003\BIN directory. * SRE2003.CFG should be in your SRE2003\CFG directory. * CHK_PRE.REX contains a few user configurable parameters (see section IV for the details). ------------------ IIc. Installing the post-response procedure Installing the post-response procedure is done by modifying the SREhttp/2 post_response configuration parameter (in PRELOADS.CFG). For example, the default PRELOADS.CFG used by SREhttp/2 has: post_response=0 You should change that to: post_response=procs\chk_post.rex Notes: * CHK_POST.REX should be in the SREHTTP2\PROCS directory. * PRELOADS.CFG should be in your SREHTTP2\CFG directory. * CHK_POST.REX contains a few user configurable parameters (see section IV for the details). ------------------ III. How does CHECK_404 work. The basic algorithim used by CHECK_404 is: a) When SRE2003 starts, a "CHK_404" cache is created. This step can be customized -- you can set the threshold number of "404" responses, and the timespan to consider when counting these responses, and a few other variables (see section II.a for details). b) On all requests, SRE2003 examines this cache (using a pre-request procedure) to see if "this client been sent too many 404 responses in the last several minutes". If this threshold has been exceeded, SRE2003 will immediately deny access to this client. This denial of access consists of either (see section IV for details): i) a simple 404 response is sent to the client (that can be modestly customize), or ii) a redirection to a "custom procedure" c) Assuming b is not binding, SREhttp/2 will process the request. If it turns out that the request yields a "404" response (a Not Found response), SREhttp/2 will do it's usual thing (such as use MTCHFILE to find possible alternatives). However, after responding, the CHECK_404 "post response" procedure will note, in the cache setup in step a, that this client has been sent a 404 response. Note that if the response was NOT a 404 response, the CHECK_404 post- response procedure does nothing. Exceptions: * If the client is one of the SUPERUSERS the post-response procedure will do nothing. That is ... SUPERUSERS are NOT subject to this 404 limit. Important detail: If the client's IP address is listed in the possibly host-specific SUPERUSER parameter of SREHTTP2.CFG, then he is NOT subject to a 404 limit. This is NOT THE SAME as having a SUPERUSER privileges assigned via an entry in USERS.CFG. * As described in section IV, you can specify a set of host_nickname for which this 404 tracking is disabled (that is, you specify hosts that will NOT be subject to 404 checking). These two exceptions are especially useful when constructing a site, a condition where missing links will probably be common. In short, CHECK_404 monitors every request, keeping track of the "404 response" status on a client specific basis. This does add extra overhead, and may slow down throughput. ------------------ IV. Specifying response options Both the pre-request (CHK_PRE.REX) and the post-response procedures (CHK_POST.REX) contain a few user-configurable options. They can be modified if you want to get fancier about what kind of responses you send to malfeasant clients, and if you want to specify hosts that don't need this protection IV.a. Parameters in CHK_PRE.REX CHK_PRE.REX (the pre-request procedure file) contains two parameters that control CHECK_404's response to a "repetitive 404" attack. REDIRECT_RESPONSE: A URL that the client will be redirected to. Typically, this will be an addon on this site, but it could be a simple HTML document (or even a URL on a different site). A "query string" will be appended to the end of this URL. The query string consits of: ?url_encoded_selector&client_ip_address Thus, if .... REDIRECT_RESPONSE=/TOO_MANY404.CMD and a client, who has exceeded his max404 threshold, at 135.22.15.22 requests /FOO/USER.DATA Then he will get a redirection response (302) with a location of /MANY404.CMD?foo/user.data&135.22.15.22 (actually, the selector is url-encoded, so the location will be /TOO_MANY404.CMD?foo%2fuser.data&135.22.15.22) If you do NOT want to redirect, set redirect_response='' -- a simple response will be sent. Example: redirect_response='/system/CHK_RED.cmd' Notes: * CHK_RED.CMD, in the SREHTTP2\ADDON\SYSTEM directory, is a simple example of an addon that will respond to clients who've gone over the limit. * Check_404 will allow unlimited accesss to the REDIRECT_RESPONSE selector. That is, when a client is over his MAX404 limit, he still can request the REDIRECT_RESPONSE selector (say, through a request after a 302 redirection response). MESSAGE_404: A short message to use when directly responding to a request from an attacker (when REDIRECT_RESPONSE=''). This string will be included in a 404 response. Example: message_404='Resource not available' Note that (assuming that REDIRECT_RESPONSE='') a 404 response is sent to attackers EVEN IF THE RESOURCE does exist. Actually, given that a client's max404 threshold has been exceeded, this response is sent by the pre-request component of CHECK_404, which doesn't even try to check on the existence of a resource corresponding to the request. IV.b. Parameters in CHK_POST.REX CHK_POST.REX (the post-response procedure file) contains one parameter that controls CHECK_404's response to a "repetitive 404" attack. NO_CHECK_HOSTS: A space delimited list of host-nicknames. All request to these hosts will NOT be checked for 404s. ------------------ V. The penalty addon Some resources are obviously off-limits, and clients who ask for it are quite likely to be ill-intentioned. You can use the "penalty addon" (CHK_PENL.CMD) to "penalize" such clients. In particular, one would createselector-specific entries in the (possibly host-specific) ATTRIBS.CFG file that identify resources that are liable to be asked for by attackers, and assign such requests a "404 request penalty". That is, each such request can be treated as a fixed number of 404 responses. One could be stringent, and set this penalty to be greater then the MAX404 threshold, or one could be more tolerant and set the penalty to two or three. Syntax: Realm: chk404_a Rule: /usr.dat Redirect: internal: /SYSTEM/chk_penl.cmd?penalty&amessage where usr.dat: an example of a file that only an attacker would look for penalty: an integer number amessage: an optional, url encoded message Another example: realm: chk404_b Rule: password.lst Alias: /SYSTEM/CHK_PENL.CMD?500&Unknown+resource In this first example, a request for PASSWORD.LST is treated harshly (since 500 is likely to be greater then any threshold you might set). Notes: * amessage is strictly optional -- it is used in a simple response to the client. Note that if included, it must be url encoded (i.e.; use + for spaces). * By default, CHK_PENL.CMD is located in SREHTTTP2\ADDON\SYSTEM