Regular Expressions are a concise
and flexible notation for finding and replacing patterns of
text.
Unless you have worked with regular
expressions before, the term and the concept may be unfamiliar
to you. However, they may not be as unfamiliar as you think.
Think about how you search for files on your hard disk. You
most likely use the ? and * characters to help find the files
you're looking for. The ? character matches a single character
in a file name, while the * matches zero or more characters.
While this method of searching for files can certainly be useful,
it is also very limited. The limited ability of the ? and *
wildcard characters give you an idea of what regular expressions
can do, but regular expressions are much more powerful and flexible.
In the regular expressions you can supply the '?' character
by '.' character and '*' character by '.*?'
Uses for Regular Expressions
In a typical search and replace operation, you must provide
the exact text you are looking for. That technique may be
adequate for simple search and replace tasks in static text,
but it lacks flexibility and makes searching dynamic text
difficult, if not impossible.
With regular expressions, you can:
Test for a pattern within a string. For example, you can
test an input string to see if a telephone number pattern
or a credit card number pattern occurs within the string.
This is called data validation.
Replace text. You can use a regular expression to identify
specific text in a document and either remove it completely
or replace it with other text.
Extract a substring from a string based upon a pattern
match. You can find specific text within a document or input
field
For example, if you need to search an entire web site to
remove some outdated material and replace some HTML formatting
tags, you can use a regular expression to test each file to
see if the material or the HTML formatting tags you are looking
for exists in that file. That way, you can narrow down the
affected files to only those that contain the material that
has to be removed or changed. You can then use a regular expression
to remove the outdated material, and finally, you can use
regular expressions to search for and replace the tags that
need replacing.
Literals
All characters are literals except: ".", "*",
"?", "+", "(", ")",
"{", "}", "[", "]",
"^" and "$". These characters are literals
when preceded by a "\". A literal is a character
that matches itself.
Wildcard
The dot character "." matches any single character.
In the HTML Batch Editor regular expressions the dot does
not match a null character.
Repeats
A repeat is an expression that is repeated an arbitrary number
of times. An expression followed by "*" can be repeated
any number of times including zero. An expression followed
by "+" can be repeated any number of times, but
at least once. An expression followed by "?" may
be repeated zero or one times only. When it is necessary to
specify the minimum and maximum number of repeats explicitly,
the bounds operator "{}" may be used, thus "a{2}"
is the letter "a" repeated exactly twice, "a{2,4}"
represents the letter "a" repeated between 2 and
4 times, and "a{2,}" represents the letter "a"
repeated at least twice with no upper limit. Note that there
must be no white-space inside the {}, and there is no upper
limit on the values of the lower and upper bounds. All repeat
expressions refer to the shortest possible previous sub-expression:
a single character; a character set, or a sub-expression grouped
with "()" for example.
Examples:
"ba*" will match all of "b", "ba",
"baaa" etc.
"ba+" will match "ba" or "baaaa"
for example but not "b".
"ba?" will match "b" or "ba".
"ba{2,4}" will match "baa", "baaa"
and "baaaa".
Non-greedy repeats
The non-greedy repeats are possible by appending a '?' after
the repeat; a non-greedy repeat is one which will match the
shortest possible string.
For example to match html tag pairs one could use something
like:
"<\s*tagname[^>]*>(.*?)<\s*/tagname\s*>"
In this case sub-expression no.1 (viz. HBESubExp,
HBEReplaceAllRE)
will contain the text between the tag pairs, and will be the
shortest possible matching string.
Parenthesis
Parentheses serve two purposes, to group items together into
a sub-expression, and to mark what generated the match.
For example the expression "(ab)*" would match all
of the string "ababab". Sub-expressions are indexed
from left to right starting from 1, sub-expression 0 is the
whole expression. Sub-expressions can be used in the HBEReplaceAllRE
in the second parameter as index number preceded by backslash.
Sub-expressions used in the HBEFindRE
can be retrieved by HBESubExp.
A following example finds a first string delimited by the
quotation marks and shows a message box window with this string.
HBEFindRE ("\"(.*?)\"")
MsgBox HBESubExp(1)
Back references
A back reference is a reference to a previous sub-expression
that has already been matched, the reference is to what the
sub-expression matched, not to the expression itself. A back
reference consists of the escape character "\" followed
by a digit "1" to "9", "\1"
refers to the first sub-expression, "\2" to the
second etc. Back references can be used in the HBEReplaceAllRE
function.
Non-Marking Parenthesis
Sometimes you need to group sub-expressions with parenthesis,
but don't want the parenthesis to spit out another marked
sub-expression, in this case a non-marking parenthesis (?:expression)
can be used. For example the following expression creates
no sub-expressions:
"(?:abc)*"
Alternatives
Alternatives occur when the expression can match either one
sub-expression or another, each alternative is separated by
a "|", or a "\|". Each alternative is
the largest possible previous sub-expression; this is the
opposite behaviour from repetition operators.
Examples:
"a(b|c)" could match "ab" or "ac".
"abc|def" could match "abc" or "def".
Sets
A set is a set of characters that can match any single character
that is a member of the set. Sets are delimited by "["
and "]" and can contain literals, character ranges,
character classes, collating elements and equivalence classes.
Set declarations that start with "^" contain the
compliment of the elements that follow.
Examples:
Character literals:
"[abc]" will match either of "a", "b",
or "c".
"[^abc] will match any character other than "a",
"b", or "c".
Character ranges:
"[a-z]" will match any character in the range "a"
to "z".
"[^A-Z]" will match any character other than those
in the range "A" to "Z".
Character classes
Character classes are denoted using the syntax "[:classname:]"
within a set declaration, for example "[[:space:]]"
is the set of all whitespace characters. The available character
classes are:
alnum
Any alpha
numeric character.
alpha
Any alphabetical character
a-z and A-Z. Other characters may also be included depending
upon the locale.
blank
Any blank
character, either a space or a tab.
cntrl
Any control character.
digit
Any digit
0-9.
graph
Any graphical character.
lower
Any lower
case character a-z. Other characters may also be included
depending upon the locale.
print
Any printable character.
punct
Any punctuation
character.
space
Any whitespace character.
upper
Any upper
case character A-Z. Other characters may also be included
depending upon the locale.
xdigit
Any hexadecimal digit character,
0-9, a-f and A-F.
word
Any word
character - all alphanumeric characters plus the underscore.
unicode
Any character whose code
is greater than 255, this applies to the wide character
traits classes only.
There are some shortcuts that can be used in place of the
character classes, provided the flag regbase::escape_in_lists
is set then you can use:
\w in place of [:word:]
\s in place of [:space:]
\d in place of [:digit:]
\l in place of [:lower:]
\u in place of [:upper:]
Line anchors
An anchor is something that matches the null string at the
start or end of a line: "^" matches the null string
at the start of a line, "$" matches the null string
at the end of a line.
Characters by code
This is an extension to the algorithm that is not available
in other libraries, it consists of the escape character followed
by the digit "0" followed by the octal character
code. For example "\023" represents the character
whose octal code is 23. Where ambiguity could occur use parentheses
to break the expression up: "\0103" represents the
character whose code is 103, "(\010)3 represents the
character 10 followed by "3". To match characters
by their hexadecimal code, use \x followed by a string of
hexadecimal digits, optionally enclosed inside {}, for example
\xf0 or \x{aff}, notice the latter example is a Unicode character.
Word operators
The following operators are provided for compatibility with
the GNU regular expression library.
"\w" matches any single character that is a member
of the "word" character class, this is identical
to the expression "[[:word:]]".
"\W" matches any single character that is not a
member of the "word" character class, this is identical
to the expression "[^[:word:]]".
"\<" matches the null string at the start of
a word.
"\>" matches the null string at the end of the
word.
"\b" matches the null string at either the start
or the end of a word.
"\B" matches a null string within a word.
Buffer operators
The following operators are provide for compatibility with
the GNU regular expression library, and Perl regular expressions:
"\`" matches the start of a buffer.
"\A" matches the start of the buffer.
"\'" matches the end of a buffer.
"\z" matches the end of a buffer.
"\Z" matches the end of a buffer, or possibly one
or more new line characters followed by the end of the buffer.
Escape operator
The escape character "\" has several meanings.
Inside a set declaration the escape character is a normal
character in which case whatever follows the escape is a literal
character regardless of its normal meaning.
The escape operator may introduce an operator for example:
back references, or a word operator.
The escape operator may make the following character normal,
for example "\*" represents a literal "*"
rather than the repeat operator.
Single character escape sequences
The following escape sequences are aliases for single characters:
Escape sequence
Character code
Meaning
\a
0x07
Bell character.
\f
0x08
Form feed.
\n
0x0A
Newline character.
\r
0x0D
Carriage return.
\t
0x09
Tab character.
\v
0x0B
Vertical tab.
\e
0x1B
ASCII Escape character.
\0dd
0dd
An octal character code, where dd is one or
more octal digits.
\xXX
0xXX
A hexadecimal character code, where XX is one or more
hexadecimal digits.
\x{XX}
0xXX
A hexadecimal character code, where XX is one or more
hexadecimal digits, optionally a unicode character.
\cZ
z-@
An ASCII escape sequence control-Z, where Z is any
ASCII character greater than or equal to the character
code for '@'.
Miscellaneous escape sequences:
The following are provided mostly for perl compatibility,
but note that there are some differences in the meanings of
\l \L \u and \U:
\w
Equivalent to [[:word:]].
\W
Equivalent to [^[:word:]].
\s
Equivalent to [[:space:]].
\S
Equivalent to [^[:space:]].
\d
Equivalent to [[:digit:]].
\D
Equivalent to [^[:digit:]].
\l
Equivalent to [[:lower:]].
\L
Equivalent to [^[:lower:]].
\u
Equivalent to [[:upper:]].
\U
Equivalent to [^[:upper:]].
\C
Any single character, equivalent to '.'.
\X
Match any Unicode combining character sequence, for
example "a\x 0301" (a letter a with an acute).
\Q
The begin quote operator, everything that follows
is treated as a literal character until a \E end quote
operator is found.
\E
The end quote operator, terminates a sequence begun
with \Q.
What gets matched?
The regular expression will match the first possible matching
string, if more than one string starting at a given location
can match then it matches the longest possible string. In
cases where their are multiple possible matches all starting
at the same location, and all of the same length, then the
match chosen is the one with the longest first sub-expression,
if that is the same for two or more matches, then the second
sub-expression will be examined and so on.
