Regular Expression Reference

This information below describes the construction and syntax of regular expressions that can be used within certain Araxis products.

The text below is an edited version of the Regex++ Library’s regular expression syntax documentation. The original text can be found on the Boost website.

Literals

All characters match themselves except for the following special characters:

.|*?+(){}[]^$\

These characters will match themselves when preceded by a \.

Wildcard

The dot character ‘.’ matches any single character.

Line anchors

A ‘^’ character matches the null string at the start of a line.

A ‘$’ character matches the null string at the end of a line.

Repeats

A repeat is an expression that is repeated an arbitrary number of times. An expression followed by ‘*’ can be repeated any number of times, including zero. An expression followed by ‘+’ can be repeated any number of times, but at least once.

An expression followed by ‘?’ may be repeated zero or one times only. When it is necessary to specify the minimum and maximum number of repeats explicitly, the bounds operator {} may be used. Thus. a{2} is the letter ‘a’ repeated exactly twice, a{2,4} represents the letter ‘a’ repeated between 2 and 4 times, and a{2,} represents the letter ‘a’ repeated at least twice with no upper limit. Note that there must be no whitespace inside the {}, and there is no upper limit on the values of the lower and upper bounds. All repeat expressions refer to the shortest possible previous sub-expression: a single character; a character set, or a sub-expression grouped with () for example.

Examples

ba* will match all of b, ba, baaa, etc.

ba+ will match ba or baaaa, for example, but not b.

ba? will match b or ba.

ba{2,4} will match baa, baaa and baaaa.

Non-greedy repeats

Non-greedy repeats are possible by appending a ‘?’ after the repeat; a non-greedy repeat is one which will match the shortest possible string.

For example, to match html element pairs, one could use:

<\s*tagname[^>]*>(.*?)<\s*/tagname\s*>

In this case $1 (tagged-match 1, from (.*?)) will contain the text between the tag pairs, and will be the shortest possible matching string.

Parenthesis

Parentheses serve two purposes, to group items together into a sub-expression, and to mark what generated the match. For example the expression (ab)* would match all of the string ababab.

When using regular expressions in Araxis Merge with a line-pairing rule, you can select which sub-expressions should be used for pairing. Likewise, when using regular expressions to ignore specific matching parts of a line rather than the entire line using an expression, you can select which sub-expressions should be ignored.

Non-marking parenthesis

Sometimes you need to group sub-expressions with parenthesis, but don’t want the parenthesis to spit out another marked sub-expression, in this case a non-marking parenthesis (?:expression) can be used. For example the following expression creates no sub-expressions:

(?:abc)*

Forward lookahead asserts

There are two forms of these; one for positive forward lookahead asserts, and one for negative lookahead asserts:

(?=abc) matches zero characters only if they are followed by the expression abc.

(?!abc) matches zero characters only if they are not followed by the expression abc.

Alternatives

Alternatives occur when the expression can match either one sub-expression or another, each alternative is separated by a ‘|’. Each alternative is the largest possible previous sub-expression; this is the opposite behaviour from repetition operators.

Examples:

a(b|c) could match ab or ac.

abc|def could match abc or def.

Character sets

A set is a set of characters that can match any single character that is a member of the set. Sets are delimited by ‘[’ and ‘]’ and can contain literals, character ranges, character classes, collating elements and equivalence classes. Set declarations that start with ‘^’ contain the compliment of (that is, everything but) the elements that follow.

Character literals

[abc] will match either of ‘a’, ‘b’, or ‘c’.

[^abc] will match any character other than ‘a’, ‘b’, or ‘c’.

Character ranges

[a-z] will match any character in the range ‘a’ to ‘z’.

[^A-Z] will match any character other than those in the range ‘A’ to ‘Z’.

Note that character ranges are highly locale dependent: they match any character that collates between the endpoints of the range. When using ranges with languages that may be affected by collation rules, we recommend trying out matching and mis-matching sample expressions to confirm that the ranges operate as intended.

Character classes

Character classes are denoted using the syntax [:classname:] within a set declaration, for example [[:space:]] is the set of all whitespace characters. The available character classes are:

There are some shortcuts that can be used in place of the character classes:

\w in place of [:word:]

\s in place of [:space:]

\d in place of [:digit:]

\l in place of [:lower:]

\u in place of [:upper:]

Collating elements

Collating elements take the general form [.tagname.] inside a set declaration, where tagname is either a single character, or a name of a collating element. For example [[.a.]] is equivalent to [a], and [[.comma.]] is equivalent to [,]. All the standard POSIX collating element names are supported, and in addition the following digraphs: ‘ae’, ‘ch’, ‘ll’, ‘ss’, ‘nj’, ‘dz’, ‘lj’, each in lower, upper and title case variations. Multi-character collating elements can result in the set matching more than one character. For example, [[.ae.]] would match two characters, but note that [^[.ae.]] would only match one character.

Equivalence classes

Equivalence classes take the general form [=tagname=] inside a set declaration, where tagname is either a single character, or a name of a collating element, and matches any character that is a member of the same primary equivalence class as the collating element [.tagname.]. An equivalence class is a set of characters that collate the same, a primary equivalence class is a set of characters whose primary sort key are all the same (for example strings are typically collated by character, then by accent, and then by case; the primary sort key then relates to the character, the secondary to the accentation, and the tertiary to the case). If there is no equivalence class corresponding to tagname, then [=tagname=] is exactly the same as [.tagname.].

To include a literal ‘-’ in a set declaration: make it the first character after the opening [ or [^, the endpoint of a range, a collating element, or precede with an escape character as in [\-]. To include a literal ‘[’ or ‘]’ or ‘^’ in a set, make them the endpoint of a range, a collating element, or precede them with an escape character.

Back references

A back reference is a reference to a previous sub-expression that has already been matched. The reference is to what the sub-expression matched, not to the expression itself. A back reference consists of the escape character ‘\’ followed by a digit ‘1’ to ‘9’. \1 refers to the first sub-expression, \2 to the second, etc. For example, the expression (.*)\1 matches any string that is repeated about its mid-point. For example, abcabc or xyzxyz. A back reference to a sub-expression that did not participate in any match matches the null string. Note that this is different to some other regular expression matchers.

Characters by code

This extension consists of the escape character followed by the digit ‘0’ followed by the octal character code. For example, \023 represents the character whose octal code is 23. Where ambiguity could occur, use parentheses to break the expression up: \0103 represents the character whose code is 103; (\010)3 represents the character 10 followed by ‘3’. To match characters by their hexadecimal code, use \x followed by a string of hexadecimal digits, optionally enclosed inside {}. For example \xf0 or \x{aff}. Notice that the latter example is a Unicode character.

Word operators

The following operators are provided for compatibility with the GNU regular expression library.

\w matches any single character that is a member of the ‘word’ character class. This is identical to the expression [[:word:]].

\W matches any single character that is not a member of the ‘word’ character class. This is identical to the expression [^[:word:]].

\< matches the null string at the start of a word.

\> matches the null string at the end of the word.

\b matches the null string at either the start or the end of a word.

\B matches a null string within a word.

The start of the sequence passed to the matching algorithms is considered to be a potential start of a word. The end of the sequence passed to the matching algorithms is considered to be a potential end of a word.

Buffer operators

The following operators are provide for compatibility with the GNU regular expression library, and Perl regular expressions:

``` matches the start of a buffer.

\A matches the start of the buffer.

\' matches the end of a buffer.

\z matches the end of a buffer.

\Z matches the end of a buffer, or possibly one or more new line characters followed by the end of the buffer.

A buffer is considered to consist of the entire line under consideration.

Escape operator

The escape character ‘\’ has several meanings.

Inside a set declaration, whatever follows the escape is a literal character regardless of its normal meaning.

The escape operator may introduce an operator, such as back references or a word operator.

The escape operator may make the following character normal. For example ‘\*’ represents a literal ‘*’ rather than the repeat operator.

Single character escape sequences

The following escape sequences are aliases for single characters:

Miscellaneous escape sequences

The following are provided mostly for perl compatibility, but note that there are some differences in the meanings of \l, \L, \u and \U:

What gets matched?

The regular expression library will match the first possible matching string. If more than one string starting at a given location can match, it matches the longest possible string. In cases where there are multiple possible matches all starting at the same location and all of them are the same length, the match chosen is the one with the longest first sub-expression. If that is the same for two or more matches, the second sub-expression will be examined and so on.