Rules
Prev | Home | Next

## Rules

When the filtering engine is enabled, every incoming request is intercepted and analysed before it is processed. The analysis begins with a series of built-in checks designed to validate the request format. These checks can be controlled using configuration directives. In the second stage, the request goes through a series of user-defined filters that are matched against the request. Whenever there is a positive match, certain actions are taken.

### Simple filtering

The most simplest form of filtering is, well, simple. It looks like this:

    SecFilter KEYWORD

For each simple filter like this, ModSecurity will look for the keyword in the request. The search is pretty broad; it will be applied to the first line of the request (the one that looks like this `GET /index.php?parameter=value HTTP/1.0`). In case of `POST` requests, the body of the request will be searched too (provided the request body buffering is enabled, of course).

### Note

All pattern matches are case insensitive by default.

### Path normalisation

Filters are not applied to raw request data, but on a normalised copy instead. We do this because attackers can (and do) apply various evasion techniques to avoid detection. For example, you might want to setup a filter that detects shell command execution:

    SecFilter /bin/sh

But the attacker may use a `string /bin/./sh` (which has the same meaning) in order to avoid the filter.

ModSecurity automatically applies the following transformations:

- On Windows only, convert `\` to `/`

- Reduce `/./` to `/`

- Reduce `//` to `/`

- Decode URL-encoded characters

You can choose whether to enable or disable the following checks:

- Verify URL encoding

- Allow only bytes from a certain range to be used

### Null byte attack prevention

Null byte attacks try to confuse C/C++ based software and trick it into thinking that a string ends before it actually does. This type of an attack is typically rejected with a proper `SecFilterByteRange` filter. However, if you do not do this a null byte can interfere with ModSecurity processing. To fight this, ModSecurity looks for null bytes during the decoding phase and converts them into spaces. So, where before this filter:

    SecFilter hidden

would not detect the word hidden in this request:

    GET /one/two/three?p=visible%00hidden HTTP/1.0

it now works as expected.

### Regular expressions

The simplest method of filtering I discussed earlier is actually slightly more complex. Its full syntax is as follows:

    SecFilter KEYWORD [ACTIONS]

First of all, the keyword is not a simple text. It is an regular expression. A regular expression is a mini programming language designed to pattern matching in text. To make most out of this (now) powerful tool you need to understand regular expressions well. I recommend that you start with one of the following resources:

- Perl-compatible regular expressions man page, http://www.pcre.org/pcre.txt

- Perl Regular Expressions, http://www.perldoc.com/perl5.6/pod/perlre.html

- Mastering Regular Expressions, http://www.oreilly.com/catalog/regex/

- Google search on regular expressions, http://www.google.com/search?q=regular%20expressions

- Wikipedia entry, http://en.wikipedia.org/wiki/Regular_expression

- POSIX regular expressions, http://www.wellho.net/regex/posix.html

### Note

Two different regular expression engines are used in Apache 1.x and Apache 2.x. The Apache 1.x regular expression engine is POSIX compliant. The Apache 2.x regular engine is PCRE compliant. As a rule of thumb, regular expressions that work in Apache 1.x will work in Apache 2.x, but not the other way round. If you need to write rules that work on both major branches you will have to test them thoroughly. Since 1.9.2 it is possible to compile ModSecurity for Apache 1.x to use PCRE as an regular expression library.

The second parameter is an action list definition, which specifies what will happen if the filter matches. Actions are explained later in this manual.

### Inverted expressions

If exclamation mark is the first character of a regular expression, the filter will treat that regular expression as inverted. For example, the following:

    SecFilter !php

will reject all requests that do not contain the word php.

### Advanced filtering

While `SecFilter` allows you to start quickly, you will soon discover that the search it performs is too broad, and doesn't work very well. Another directive:

    SecFilterSelective LOCATION KEYWORD [ACTIONS]

allows you to choose exactly where you want the search to be performed. The `KEYWORD` and the `ACTIONS` bits are the same as in `SecFilter`. The `LOCATION` bit requires further explanation.

The `LOCATION` parameter consist of a series of location identifiers separated with a pipe.

Examine the following example:

    SecFilterSelective "REMOTE_ADDR|REMOTE_HOST" KEYWORD

It will apply the regular expression only the IP address of the client and the host name. The list of possible location identifiers includes all CGI variables, and some more. Here is the full list:

- `REMOTE_ADDR`

- `REMOTE_HOST`

- `REMOTE_USER`

- `REMOTE_IDENT`

- `REQUEST_METHOD`

- `SCRIPT_FILENAME`

- `PATH_INFO`

- `QUERY_STRING`

- `AUTH_TYPE`

- `DOCUMENT_ROOT`

- `SERVER_ADMIN`

- `SERVER_NAME`

- `SERVER_ADDR`

- `SERVER_PORT`

- `SERVER_PROTOCOL`

- `SERVER_SOFTWARE`

- `TIME_YEAR`

- `TIME_MON`

- `TIME_DAY`

- `TIME_HOUR`

- `TIME_MIN`

- `TIME_SEC`

- `TIME_WDAY`

- `TIME`

- `API_VERSION`

- `THE_REQUEST`

- `REQUEST_URI`

- `REQUEST_FILENAME`

- `REQUEST_BASENAME`

- `IS_SUBREQ`

There are some special locations:

- `POST_PAYLOAD` – filter the body of the `POST` request

- `ARGS` - filter arguments, the same as `QUERY_STRING|POST_PAYLOAD`

- `ARGS_NAMES` – variable/parameter names only

- `ARGS_VALUES` – variable/parameter values only

- `COOKIES_NAMES` - cookie names only

- `COOKIES_VALUES` - cookie values only

- `SCRIPT_UID`

- `SCRIPT_GID`

- `SCRIPT_USERNAME`

- `SCRIPT_GROUPNAME`

- `SCRIPT_MODE`

- `SCRIPT_BASENAME`

- `ARGS_COUNT`

- `COOKIES_COUNT`

- `HEADERS`

- `HEADERS_COUNT`

- `HEADERS_NAMES`

- `HEADERS_VALUES`

- `FILES_COUNT`

- `FILES_NAMES`

- `FILES_SIZES`

And even more special:

- `HTTP_header – search request header "header"` (`HEADER_header` also works as of 1.9)

- `ENV_variable` – search environment variable `variable`

- `ARG_variable` – search request variable/parameter `variable`

- `COOKIE_name` - search cookie with name `name`

- `FILE_NAME_variable` - search the filename of the file uploaded under the name `variable`.

- `FILE_SIZE_variable` - search the size of the file uploaded under the name `variable`

A limited number of output-specific variables are also available for Apache 2 (only when output buffering is enabled):

- OUTPUT - the complete response body

- OUTPUT_STATUS - response status code

### Argument filtering exceptions

The `ARG_variable` location names support inverted usage when used together with the `ARG` location. For example:

    SecFilterSelective "ARGS|!ARG_param" KEYWORD

will search all arguments except the one named param.

### Cookies

ModSecurity provides full support for Cookies. By default cookies will be treated as if they were in version 0 format (Netscape-style cookies). However, version 1 cookies (as defined in RFC 2965) are also supported. To enable version 1 cookie support use the `SecFilterCookieFormat` directive:

    # enable version 1 (RFC 2965) cookies
    SecFilterCookieFormat 1

By default, ModSecurity will not try to normalise cookie names and values. However, since some applications and platforms (e.g. PHP) do encode cookie content you can choose to apply normalisation techniques to cookies. This is done using the `SecFilterNormalizeCookies` directive.

    SecFilterNormalizeCookies On

### Note

Prior to version 1.8.7 ModSecurity supported the `SecFilterCheckCookieFormat` directive. Due to recent changes in 1.8.7 this directive is now deprecated. It can still be used in the configuration but it does not do anything. The directive will be completely removed in the 1.9.x branch.

### Output filtering

ModSecurity supports output filtering in the version for Apache 2. It is disabled by default so you need to enable it first:

    SecFilterScanOutput On

After that, simply add selective filters using a special variable `OUTPUT`:

    SecFilterSelective OUTPUT "credit card numbers"

Those who have perhaps followed my columns at http://www.webkreator.com/php/ know that I am somewhat obsessed with the inability of PHP to prevent fatal errors. I have gone to great lengths to prevent fatal errors from spilling to end users (see http://www.webkreator.com/php/configuration/handling-fatal-and-parse-errors.html) but now, finally, I don't have to worry any more about that. The following will catch PHP output error in the response body, replace the response with an error, and execute a custom PHP script (so that the application administrator can be notified):

    SecFilterSelective OUTPUT "Fatal error:" deny,status:500
    ErrorDocument 500 /php-fatal-error.html

You should note that although you can mix output filters with input filters, they are not executed at the same time. Input filters are executed before a request is processed by Apache, while the output filters are executed after Apache completes request processing.

### Note

Actions `skipnext` and `chain` do not work with output filters.

Output filtering is only useful for plain text and HTML output. Applying regular expressions to binary content (for example images) will only slow down the server. By default ModSecurity will scan output in responses that have no content type, or whose content type is `text/plan` or `text/html`. You can change this using the `SecFilterOutputMimeTypes` directive:

    SecFilterOutputMimeTypes "(null) text/html text/plain"

Configured as in example above ModSecurity will apply output filters to plain text files, HTML files, and files where the mime type is not specified "(null)".

### Note

Using output buffering will make ModSecurity keep the whole of the page output in memory, no matter how large it is. The memory consumption is over twice the size of the page length.

While output monitoring is a useful feature in some circumstances you should be aware that it isn't foolproof. If an attacker is in a full control of request processing she can evade output monitoring in two ways:

1. Use a Content-Type that is not being monitored. (For performance reasons it is not feasible to monitor all content types.)

2. Encode the output in some way. Any simple encoding is likely to be enough to fool monitoring.

As of 1.9 another output variable is supported - `OUTPUT_STATUS`. This variable contains the status code of the response.

Prev |  | Next
Configuration | Home | Actions

Copyright (C) 2002-2006 Thinking Stone