Technical Outline of SREhttp/2

Technical outline of SREhttp/2

This documents outlines the steps taken by SREhttp/2 when processing a request. Note that several of these steps are done by SRE2003 -- they are not dependent on the use of the SREhttp/2 filter.

The steps:

Connect	Check_limits	Pre-Transaction	Pre-Request	Check_cache
Fast-Filter	SREhttp/2_invoked	Access_control	Client-specific attributes	Redirection_lookup
EXECs	!Special	30x_redirect	Literal_assignment	Create_response
Pre-Response	Send_response	IM and Range	GZIP_as_TE	Post-response

Notes:

Response_modifications

Content-type

EXEC,
pre-response procedures,
and resource existence

SSIs

CGI and addons

`Client connection`		Client connects by issuing a request to the port that SRE2003 is listening to
`Check limits and bad IP addresses`		SRE2003 makes sure that the server is not too busy, and that the client is not explicitily UNALLOWED
`SRE2003 "pre-transaction" procedure`		Optional procedures executed by SRE2003 before reading request information (but after the client has established a connection).
`SRE2003 "pre-request" procedure`		Optional procedures executed by SRE2003 after reading request information.
`Check request cache`		Check the SRE2003 request cache for a useable cached version of this request. If one is found (and returned to the client), the filter may be called anyways.
`The fast-filter`		Optional small and fast filter called before SREhttp/2 is invoked.
`The filter`		SREhttp/2 is called.
	`Access control and selector specific attributes`	Selector-specific attributes are looked up. If necessary, client privileges may be compared against required privileges. This can involve sending an authorization response, followed by username lookup.
	`Client-specific attributes`	If a client advanced-option has been specified -- then client privilege information can be used to modify the `failure, redirect, permissions`, and the `advanced options` selector-specific attributes.
	`Redirection and other selector attributes`	One of the following may be done: Internal redirection (remapping of the request selector to a different request selector). Literal assignments are resolved. Temporary,permanent , and notify redirection (corresponding to http response codes 302, 301, and 200). content negotiation. If content-negotiation is selected, a custom procedure can be specified. Then, if either internal redirection or no redirection.... ~ replacement may be done appending a trailing / to a link to a directory If content negotiation, access control and username lookup may be repeated (given that content negotiation results in choice of a variant to use). The end result is either a modified-selector for internal use, a modified selector to redirect to, or the name of a file to send to the client. However, no actual redirection occurs at this step.
	`SREhttp/2 EXEC selector- specific procedures`	Optional selector-specific procedures can be called. These may (though they usually do not) send a response to the client. [in which case, skip to `SRE2003 "IM" procedures`]
	`!Special` commands	If a !Special command was requested, process it and return the results. [in which case, skip to `SREhttp/2 post-response procedure`]
	`Redirection responses`	If a redirection (TEMP, MOVE, or NOTIFY) has been specified, or appending a trailing / occured, then redirection occurs now (before the pre-response procedure). [in which case, skip to `SREhttp/2 post-response procedure`]
	`Literal` assignment	If literal assignments was specified, the specified file is sent to the client. [in which case, skip to `SRE2003 "IM" procedures`]. Note: if HTACCESS controls are enabled, check the appropriate HTACCESS file before granting access to this resource.
	`Non-standard methods`	Process PUT, DELETE, and other non-standard http requests. If this happens, skip to SREhttp/2 post-response procedure.
	`Create a response to send to client`	If HTACCESS controls are enabled, and a file is being returned (such as a html file with SSIs), then first check the appropriate HTACCESS file before granting access. If access is denied, skip to the post-response step. Read from a disk, or dynamically create, a response. This may be an html document (possibly with server side includes), a text or binary file, or the output of an SREhttp/2 addon or cgi-bin script. If the request is for a non-existent resource (say, for a non-existent text file), then a 404 response is immediately sent: Either a generic, or a A NOT_FOUND_FILE, is used as the text of a 404 response A pre-response is not* called with 404 responses.*
	`SREhttp/2 "pre-response" procedure`	Given the output from the prior step (and given that a 404 response was not sent), SREhttp/2 can than apply an optional pre-response procedure to the contents of the response. For example, adding a date stamp to html documents; or performing on-the-fly content-encoding; or applying encryption. Note that output from !SPECIAL commands, from CGI-BIN scripts, and from many SREhttp/2 addons, will not be modified by pre-response procedures.
	`Send the response to the client`	The output, perhaps as modified by a pre-response procedure, is then sent to the client. If needed, a MD5 hash is generated (using this output). In technical terms, this is the instance, and will be identified by a unique to this URI etag. Thus, although the contents of the instance may not change, what is actually sent to the client may be the result of further processing, including `Instance Manipulation`, `Range Extraction`, and `Transfer Encoding`.
`SRE2003 "IM" procedures and range responses`		Possibly do a range extraction. Also, possibly execute Instance Manipulation procedures Instance manipulation includes sophisticated range extraction, and delta encoding. Basically, instance manipulation does not change the underlying content of a response (the instance), though it may change the bytes used to represent this content.
`On the fly GZIP encoding as a transfer encoding`		If the client (or, a proxy between you and the client) understands `GZIP as transfer encoding`, then SRE2003 can GZIP on-the-fly.
`Transmit the response`		The actual bytes of the response, including response header and response body, are sent by SRE2003. SREhttp/2 can wait for conclusion of this transmission, or it can assume that SRE2003 will succeed and immediately proceed to the next step.
	`Auditing and SREhttp/2 "post-response" procedure`	Log information can be written to several common log files. A running count of the number of requests, by URI, can also be augmented. Optional procedure called after a response has been sent to the client. For example, you can use post-response procedures for advanced auditing, or send event specific email alerts. The NO_POSTRESPONSE permission can be used to suppress use of a post-response procedure.

Notes

Sending a file to a client	When SREhttp/2 sends a file to a client, several general and selector-specific parameters are checked. These include: Specifying a selector specific mimetype Adding selector-specific headers and response code Generating a Content-MD5 response header Permitting on-the-fly GZIP (as a transfer-encoding) Allowing use of range extraction
Determinine the content-type	When sending a static file to a client, SREhttp/2 determines the mimetype (which is then used in a `Content-Type` response header) in several steps... Check for a SENDAS_mime_submime special directive Check for a selector specific mime type assignment Check the list of custom extension-to-mimetype mappings Check the (short) list of hardcoded extension-to-mimetype mappings Note that you can switch the order of 2 and 3 (that is, check the hardcoded mappings first) by setting the CHECK_MIMETYPE_FILE parameter.
EXEC, pre-response procedures, and resource exixtence	`EXEC`s are done even if the resource is not available. For example, they are done before SREhttp/2 checks to see if a requested HTML file actually exists. This means that EXECs will be called even on `404 (Not Found)` responses. `Pre-response` procedures are not called if a resource does not exist -- they are only called just prior to sending the contents of a resource (such as a static file, or an html document with SSIs). If there is no such resource (that is, on `404` responses), then pre-response procedures are not called.
Processing server side includes	When an HTML file is identified as being subject to server side includes (`SSIs`), a number of SSI keyphrases can be processed.
SREhttp/2 addons and CGI-BIN scripts	SREhttp/2 can dynamically create content through the use of CGI-BIN scripts, or through SREhttp/2 addons. Note that when using either of these methods, the resulting content may not be subject to pre-resonse procedures.
To Do	Feel free to comment on our working list of feature to add to SREhttp/2.