re2: R interface to the Google's RE2 (C++) regular-expression library

Description

Regular expression matching can be done in two ways: using recursive backtracking or using finite automata-based techniques.

Perl, PCRE, Python, Ruby, Java, and many other languages rely on recursive backtracking for their regular expression implementations. The problem with this approach is that performance can degrade very quickly. Time complexity can be exponential. In contrast, re2 uses finite automata-based techniques for regular expression matching, guaranteeing linear time execution and a fixed stack footprint. See links to Russ Cox's excellent articles in references section.

re2 supports pearl style regular expressions (with extensions like \d, \w, \s, ...) and provides most of the functionality of PCRE – eschewing only backreferences and look-around assertions.

Primary re2 functions

re2 supports three types of operations on a character vector: matching (substring extraction), detection, and replacement.

Matching and substring extraction is provided by re2_match and re2_match_all. Matching regexp "(foo)|(bar)baz" on "barbazbla" will return submatches '.0' = "barbaz", '.1' = NA, and '.2' = "bar". '.0' is the entire matching text. '.1' is the first group, and so on. Groups can also be named.

re2_detect finds the presence of a pattern in a string, like grepl of base R.

re2_replace and re2_replace_all substitute matched substring with replacement string. Replacing first occurrence of pattern "b+" using replacement string "d" on text "yabba dabba doo" will result in "yada dabba doo". Replacing globally will result in "yada dada doo". re2_extract_replace functions like re2_replace except that non-matching text is ignored (not returned).

In all the above functions regexp patterns can be pre-compiled and reused. This greatly improves performance when the same regular-expression pattern is used repeatedly. See re2_regexp.

List of re2 functions :

• re2_match

• re2_match_all

• re2_split

• re2_detect

• re2_which

• re2_subset

• re2_locate

• re2_locate_all

• re2_count

• re2_replace

• re2_replace_all

• re2_extract_replace

• re2_regexp

• re2_get_options

Author(s)

Girish Palya <girishji@gmail.com>

References

• Regular Expression Matching Can Be Simple And Fast https://swtch.com/~rsc/regexp/regexp1.html

• Regular Expression Matching: the Virtual Machine Approach https://swtch.com/~rsc/regexp/regexp2.html

• Regular Expression Matching in the Wild https://swtch.com/~rsc/regexp/regexp3.html

• RE2 Syntax https://github.com/google/re2/wiki/Syntax

• RE2 C++ source https://github.com/google/re2

• R source of RE2 https://github.com/girishji/re2

See Also

Useful links:

• https://github.com/girishji/re2

• Report bugs at https://github.com/girishji/re2/issues

Count the number of matches in a string

Description

Vectorized over string and pattern. Match against a string using a regular expression and return the count of matches.

Usage

re2_count(string, pattern)

Arguments

Value

An integer vector.

See Also

re2_regexp for options to regular expression, re2_syntax for regular expression syntax.

Examples

color <- c("yellowgreen", "steelblue", "goldenrod", "forestgreen")
re2_count(color, "e")
re2_count(color, "r")

# Regular expression vs literal string
re2_count(c("..", "a...", "foo.b"), ".")
re2_count(c("..", "a...", "foo.b"), re2_regexp(".", literal = TRUE))

Find the presence of a pattern in string(s)

Description

Equivalent to grepl(pattern, x). Vectorized over string and pattern. For the equivalent of grep(pattern, x) see re2_which.

Usage

re2_detect(string, pattern)

Arguments

Value

A logical vector. TRUE if match is found, FALSE if not.

See Also

re2_regexp for options to regular expression, re2_syntax for regular expression syntax, and re2_match to extract matched groups.

Examples

## Character vector input
s <- c("barbazbla", "foobar", "not present here ")
pat <- "(foo)|(bar)baz"
re2_detect(s, pat)

## Use precompiled regexp
re <- re2_regexp("(foo)|(bAR)baz", case_sensitive = FALSE)
re2_detect(s, re)

Extract with substitutions

Description

Like re2_replace, except that if the pattern matches, "rewrite" string is returned with substitutions. The non-matching portions of "text" are ignored.

Difference between re2_extract_replace and re2_replace:

> re2_extract_replace("bunny@wunnies.pl", "(.*)@([^.]*)", "\2!\1")
[1] "wunnies!bunny"

> re2_replace("bunny@wunnies.pl", "(.*)@([^.]*)", "\2!\1")
[1] "wunnies!bunny.pl"

"\1" and "\2" are names of capturing subgroups.

Vectorized over string and pattern.

Usage

re2_extract_replace(string, pattern, rewrite)

Arguments

Value

A character vector with extractions.

See Also

re2_regexp for options to regular expression, re2_syntax for regular expression syntax. See re2_replace and re2_replace_all to replace pattern in place.

Examples

# Returns extracted string with substitutions
re2_extract_replace(
  "bunny@wunnies.pl",
  "(.*)@([^.]*)",
  "\\2!\\1"
)

# Case insensitive
re2_extract_replace(
  "BUNNY@wunnies.pl",
  re2_regexp("(b.*)@([^.]*)", case_sensitive = FALSE),
  "\\2!\\1"
)

# Max submatch too large (1 match group, 2 submatches needed).
#   Replacement fails and empty string is returned.
re2_extract_replace("foo", "f(o+)", "\\1\\2")

Retrieve options

Description

re2_get_options returns a list of all options from a RE2 object (internal representation of compiled regexp).

Usage

re2_get_options(re2ptr)

Arguments

Value

A list of options and their values.

See Also

re2_regexp.

Locate the start and end of pattern in a string

Description

Vectorized over string and pattern. For matches of 0 length (ex. spatial patterns like "$") end will be one character greater than beginning.

Usage

re2_locate(string, pattern)

re2_locate_all(string, pattern)

Arguments

Value

re2_locate returns an integer matrix, and re2_locate_all returns a list of integer matrices.

See Also

re2_regexp for options to regular expression, re2_syntax for regular expression syntax.

Examples

color <- c("yellowgreen", "steelblue", "goldenrod", "forestgreen")

re2_locate(color, "$")
re2_locate(color, "l")
re2_locate(color, "e")

# String length can be a multiple of pattern length
re2_locate(color, c("l(l|d)?", "st"))

# Locate all occurrences
re2_locate_all(color, "l")
re2_locate_all(color, "e")

# Locate all characters
re2_locate_all(color, ".")

Extract matched groups from a string

Description

Vectorized over string and pattern. Match against a string using a regular expression and extract matched substrings. re2_match extracts first matched substring, and re2_match_all extracts all matches.

Matching regexp "(foo)|(bar)baz" on "barbazbla" will return submatches '.0' = "barbaz", '.1' = NA, and '.2' = "bar". '.0' is the entire matching text. '.1' is the first group, and so on. Groups can also be named.

Usage

re2_match(string, pattern, simplify = TRUE)

re2_match_all(string, pattern)

Arguments

Value

In case of re2_match a character matrix. First column is the entire matching text, followed by one column for each capture group. If simplify is FALSE, returns a list of named character vectors.
In case of re2_match_all, returns a list of character matrices.

See Also

re2_regexp for options to regular expression, re2_syntax for regular expression syntax.

Examples

## Substring extraction
strings <- c("barbazbla", "foobar")
pattern <- "(foo)|(?P<TestGroup>bar)baz"

re2_match(strings, pattern)
result <- re2_match(strings, pattern)
is.matrix(result)

re2_match(strings, pattern, simplify = FALSE)
result <- re2_match(strings, pattern, simplify = FALSE)
is.list(result)

## Compile regexp
re <- re2_regexp("(foo)|(BaR)baz", case_sensitive = FALSE)
re2_match(strings, re)

strings <- c(
  "Home: 743 733 5365", "373-733-5753 ", "foobar",
  "733.335.3457 and Work: 573-433-7577 "
)
re <- re2_regexp("([0-9]{3})[- .]([0-9]{3})[- .]([0-9]{4})")
re2_match(strings, re)

## Vectorized over patterns
re2_match(strings, c(re, "53 $", "^foo", re))

## Match all occurances, not just the first
re2_match_all(strings, re)
re2_match_all("ruby:1234 68 red:92 blue:", "(\\w+):(\\d+)")

## Vectorized over patterns (matching all occurances)
re2_match_all(strings, c(re, "53 $", "^foo", re))

Compile regular expression pattern

Description

re2_regexp compiles a character string containing a regular expression and returns a pointer to the object.

Usage

re2_regexp(pattern, ...)

Arguments

Value

Compiled regular expression.

Regexp Syntax

RE2 regular expression syntax is similar to Perl's with some of the more complicated things thrown away. In particular, backreferences and generalized assertions are not available, nor is ⁠\Z⁠.

See re2_syntax for the syntax supported by RE2, and a comparison with PCRE and PERL regexps.

For those not familiar with Perl's regular expressions, here are some examples of the most commonly used extensions:

The double backslashes are needed when writing R string literals. However, they should NOT be used when writing raw string literals:

When using UTF-8 encoding, case-insensitive matching will perform simple case folding, not full case folding.

See Also

re2_syntax has regular expression syntax.

Examples

re2p <- re2_regexp("hello world")
stopifnot(mode(re2p) == "externalptr")

## UTF-8 and matching interface
# By default, pattern and input text are interpreted as UTF-8.
# The Latin1 option causes them to be interpreted as Latin-1.
x <- "fa\xE7ile"
Encoding(x) <- "latin1"
re2_detect(x, re2_regexp("fa\xE7", encoding = "Latin1"))

## Case insensitive
re2_detect("fOobar ", re2_regexp("Foo", case_sensitive = FALSE))

## Literal string (as opposed to regular expression)
## Matches only when 'literal' option is TRUE
re2_detect("foo\\$bar", re2_regexp("foo\\$b", literal = TRUE))
re2_detect("foo\\$bar", re2_regexp("foo\\$b", literal = FALSE))

## Use of never_nl
re <- re2_regexp("(abc(.|\n)*def)", never_nl = FALSE)
re2_match("abc\ndef\n", re)
re <- re2_regexp("(abc(.|\n)*def)", never_nl = TRUE)
re2_match("abc\ndef\n", re)

Replace matched pattern in string

Description

re2_replace replaces the first match of "pattern" in "string" with "rewrite" string.

  re2_replace("yabba dabba doo", "b+", "d")

will result in "yada dabba doo".

re2_replace_all replaces successive non-overlapping occurrences of "pattern" in "text" with "rewrite" string, or performs multiple replacements on each element of string.

  re2_replace_all("yabba dabba doo", "b+", "d")
  re2_replace_all(c("one", "two"), c("one" = "1", "1" = "2", "two" = "2"))

will result in "yada dada doo".
Replacements are not subject to re-matching. Because ⁠re2_replace_all⁠ only replaces non-overlapping matches, replacing "ana" within "banana" makes only one replacement, not two.

Vectorized over string and pattern.

Usage

re2_replace(string, pattern, rewrite)

re2_replace_all(string, pattern, rewrite = "")

Arguments

Value

A character vector with replacements.

See Also

re2_regexp for options to regular expression, re2_syntax for regular expression syntax.

Examples

string <- c("yabba dabba doo", "famabbb sb")
re2_replace(string, "b+", "d")
re2_replace_all(string, "b+", "d")

# Rearrange matching groups in replaced string
re2_replace(
  "boris@kremvax.ru",
  "(.*)@([^.]*)", "\\2!\\1"
)

# Use complied pattern
string <- "the quick brown fox jumps over the lazy dogs."
re <- re2_regexp("(qu|[b-df-hj-np-tv-z]*)([a-z]+)")
rewrite <- "\\2\\1ay"
re2_replace(string, re, rewrite)
re2_replace_all(string, re, rewrite)

string <- "abcd.efghi@google.com"
re <- re2_regexp("\\w+")
rewrite <- "\\0-NOSPAM"
re2_replace(string, re, rewrite)
re2_replace_all(string, re, rewrite)

string <- "aba\naba"
re <- re2_regexp("a.*a")
rewrite <- "(\\0)"
re2_replace(string, re, rewrite)
re2_replace_all(string, re, rewrite)

# Vectorize string and pattern
string <- c("ababababab", "bbbbbb", "bbbbbb", "aaaaa")
pattern <- c("b", "b+", "b*", "b*")
rewrite <- "bb"
re2_replace(string, pattern, rewrite)
re2_replace_all(string, pattern, rewrite)

Split string based on pattern

Description

Vectorized over string and pattern.

Usage

re2_split(string, pattern, simplify = FALSE, n = Inf)

Arguments

Value

A list of string vectors or a string matrix. See option.

See Also

re2_regexp for options to regular expression, re2_syntax for regular expression syntax, and re2_match to extract matched groups.

Examples

panagram <- c(
  "The quick brown fox jumps over the lazy dog",
  "How vexingly quick daft zebras jump!"
)

re2_split(panagram, " quick | over | zebras ")
re2_split(panagram, " quick | over | zebras ", simplify = TRUE)

# Use compiled regexp
re <- re2_regexp("quick | over |how ", case_sensitive = FALSE)
re2_split(panagram, re)
re2_split(panagram, re, simplify = TRUE)

# Restrict number of matches
re2_split(panagram, " quick | over | zebras ", n = 2)

RE2 Regular Expression Syntax

Description

The simplest regular expression is a single literal character. Except for the metacharacters like *+?()|, characters match themselves. To match a metacharacter, escape it with a backslash: \+ matches a literal plus character.

Two regular expressions can be alternated or concatenated to form a new regular expression: if e_1 matches s and e_2 matches t, then e_1|e_2 matches s or t, and e_1e_2 matches st.

The metacharacters *, +, and ? are repetition operators: e_1* matches a sequence of zero or more (possibly different) strings, each of which match e_1; e_1+ matches one or more; e_1? matches zero or one.

The operator precedence, from weakest to strongest binding, is first alternation, then concatenation, and finally the repetition operators. Explicit parentheses can be used to force different meanings, just as in arithmetic expressions. Some examples: ab|cd is equivalent to (ab)|(cd); ab* is equivalent to a(b*).

The syntax described so far is most of the traditional Unix egrep regular expression syntax. This subset suffices to describe all regular languages: loosely speaking, a regular language is a set of strings that can be matched in a single pass through the text using only a fixed amount of memory. Newer regular expression facilities (notably Perl and those that have copied it) have added many new operators and escape sequences, which make the regular expressions more concise, and sometimes more cryptic, but usually not more powerful.

This page lists the regular expression syntax accepted by RE2. It also lists some syntax accepted by PCRE, PERL, and VIM.

Implementation restriction: The counting forms x{n,m}, x{n,}, and x{n} reject forms that create a minimum or maximum repetition count above 1000. Unlimited repetitions are not subject to this restriction.

Flag syntax is xyz (set) or -xyz (clear) or xy-z (set xy, clear z).

Select strings that match, or find their positions

Description

re2_subset returns strings that match a pattern. re2_which is equivalent to grep(pattern, x). It returns position of string that match a pattern. Vectorized over string and pattern. For the equivalent of grepl(pattern, x) see re2_detect.

Usage

re2_which(string, pattern)

re2_subset(string, pattern)

Arguments

Value

re2_subset returns a character vector, and re2_which returns an integer vector.

See Also

re2_regexp for options to regular expression, re2_syntax for regular expression syntax, and re2_detect to find presence of a pattern (grep).

Examples

color <- c("yellowgreen", "steelblue", "GOLDENROD", "forestgreen")
re2_which(color, "o")
re2_subset(color, "o")

re2_which(c("x", "y", NA, "foo", ""), ".")
re2_subset(c("x", "y", NA, "foo", ""), ".")

# Use precompiled regexp
re <- re2_regexp("[a-z]")
re2_which(color, re)
re2_subset(color, re)

re <- re2_regexp("[a-z]", case_sensitive = FALSE)
re2_which(color, re)
re2_subset(color, re)

# Vector of patterns
re2_which(color, c("^o", "bl.e$", re, "$"))