9. Regular Expression (Regex)

Regular Expression (Regex)

Regular Expression is another Domain Specific Language (DSL) for matching and extracting sub sequences from the main sequence based on patterns
A regular expression denotes a pattern which consists of the following operators/sub expressions:
1. Literal (A literal denotes a single symbol or character )
2. Concatenation (A concatenation denotes a sequence of patterns)
3. Choice (A choice denotes an alternative of patterns)
4. Repetition (A repetition denotes a repeat of patterns)
5. Anchor (Anchor ^ marks that a pattern must match the beginning of the line, $ marks that a pattern must match till the end of the line)
6. Character Class (Allows a quick and easy definition of a choice of set of character)
7. Capture Group (Allow extract sub-part from the matching string )
8. Non-captured Group (Allow sub expression to be group to gather for disambiguation, without being capture )
9. Escaped character (Some character has special meaning in regex, to remove the special meaning)

1. Literal

A literal denotes a single symbol or character
- Match beginning of the string must match this regrex
- Search scan the entire string to look for the regrex (partial match)
- A special literal "." matches any single character

import re

def check_match(mObj):

    if mObj is not None:

        matched = mObj.group(0)

        print(matched)

    else:

        print("Not matched!")

s = "y auu"

p = re.compile("a")

check_match(p.match(s))

Not matched!

y = "ab a"

p = re.compile("a")

check_match(p.match(y))

import re

s="foo bar baz quux"

p = re.compile( r'b\w{2}')

check_match(p.match(s))

check_match(p.search(s))

Not matched!

bar

import re

o="foo bar baz quux"

p = re.compile( r'q\w{3}')

check_match(p.match(o))

check_match(p.search(o))

Not matched!

quux

p = re.compile(".") # Special literal "." matches any single character

h = re.compile ("\w") #\w stands for "word characters"

s = "hk"

check_match(p.match(s))

check_match(h.match(s))

2. Concatenation

A concatenation denotes a sequence of patterns

s = "ab"

p = re.compile("ab")

check_match(p.match(s))

p = re.compile("ab")

check_match(p.match("abc"))

p = re.compile("ab")

check_match(p.match("aab"))

Not matched!

p = re.compile("ant")

check_match(p.match("an"))

check_match(p.match("ant"))

check_match(p.match("anti-social"))

check_match(p.match("fantasy"))

check_match(p.search ("fantasy"))

Not matched!

ant

Not matched!

ant

3. Choice

A choice denotes an alternative of patterns

s = "segob"

p = re.compile("\w+a|\w+b") # words ending with "a" or "c"

check_match(p.match(s))

check_match(p.match("paa"))

check_match(p.match("abc"))

check_match(p.match("nndhddndhdcb"))

segob

paa

nndhddndhdcb

p = re.compile("aa|b")

check_match(p.match("aa"))

check_match(p.match("b"))

check_match(p.match("aab"))

check_match(p.match("ab"))

Not matched!

4. Repetition

A repetition denotes a repeat of patterns
- * repeats 0 or more
- + repeats 1 or more
- ? repeats 0 or once
- {m,n} repeats at least m times and maximum n times. When m is omitted, the minimum is 0, when n is omitted, the maximum is unlimited

s = "aaaa"

p = re.compile("a*")

check_match(p.match(s))

aaaa

p = re.compile("a*") # * repeats 0 or more

check_match(p.match("a"))

check_match(p.match("aaaaaa"))

check_match(p.match("ab"))

check_match(p.match("ba")) # return ""

check_match(p.match("bb")) # return ""

aaaaaa

p = re.compile("a+") # + repeats 1 or more

check_match(p.match("a"))

check_match(p.match("aaaaaa"))

check_match(p.match("ab"))

check_match(p.match("ba"))

check_match(p.match("bb"))

aaaaaa

Not matched!

p = re.compile("a?") # ? repeats 0 or once

check_match(p.match("a"))

check_match(p.match("aaaaaa"))

check_match(p.match("ab"))

check_match(p.match("ba")) # return ""

check_match(p.match("bb")) # return ""

p = re.compile("a{2,3}") #{m,n} repeats at least m times and maximum n times

check_match(p.match("a"))

check_match(p.match("aaaaaa"))

check_match(p.match("ab"))

check_match(p.match("ba"))

check_match(p.match("bb"))

Not matched!

aaa

Not matched!

5. Anchor

Anchor ^ marks that a pattern must match the beginning of the line
$ marks that a pattern must match till the end of the line

s = "baaaaaaaa"

p2 = re.compile("^a+") # To make the search() to behave likes match(), we add the ^ to the beginning

check_match(p2.search(s))

Not matched!

s = "baaaaaaaa"

p2 = re.compile("a+")

check_match(p2.search(s))

aaaaaaaa

p = re.compile("\w+ab$")

check_match(p.match("oooab"))

check_match(p.match("ooocc"))

oooab

Not matched!

p = re.compile("^fan") # beginning of the line

check_match(p.search("fantasy"))

p = re.compile("asy$") # end of the line

check_match(p.search("fantasy"))

fan

asy

6. Character Class

Allows a quick and easy definition of a choice of set of character
\W denotes a set of white space characters
\d denotes a set of digit characters
[^...] denotes the negation of the character class in [...]

s = "abcdefghijklmnopqrstuvwxyz"

p = re.compile("[a-z]*")

check_match(p.match(s))

abcdefghijklmnopqrstuvwxyz

s = "abcdefghijklmnopqrstuvwxyz"

s="acbdbecmnop"

p = re.compile("[a-c]*")

check_match(p.match(s))

acb

s = "d2ebAfghicj1klmnoPqrs4tuvwxyz"

p = re.compile("[a-zA-Z0-9]*")

check_match(p.match(s))

d2ebAfghicj1klmnoPqrs4tuvwxyz

7. Capture Group

Allow extract sub-part from the matching string

p = re.compile("^[sS]([0-9]{7,7})[a-zA-Z]$")

m = p.match("s1234567Z")

if m is not None:

  print(m.group)

  print(m.group(0))

  print(m.group(1))

<built-in method group of re.Match object at 0x00000188F2DAA730>

s1234567Z

1234567

8. Non-captured Group

Allow sub expression to be group to gather for disambiguation, without being capture

s="ftp://stackoverflow.com/my.html"

p = re.compile("(?:https|ftp)://([^/\r\n]+)(/[^\r\n]*)?")

m = p.match(s)

if m is not None:

  print(m.group)

  print(m.group(0))

  print(m.group(1))

  print(m.group(2))

<built-in method group of _sre.SRE_Match object at 0x7f71edb540b8>

ftp://stackoverflow.com/my.html stackoverflow.com

/my.html

9. Escaped character

Some character has special meaning in regex, to remove the special meaning:
- E.g. ., \, -, [
To remove the special meaning, put an additional \ in front of the special character

p = re.compile("[0-9\-\+\(\) ]+")

#s = "(+65) 6550-1696"

s = "6550-1696(+65)"

m = p.match(s)

check_match(m)

6550-1696(+65)

Singapore Residential Addresses Examples

res_addr = "[Bb]lock [0-9]{1,3}[a-zA-Z]? #[0-9]{1,2}\-[0-9]{2,4} [A-Za-z ]+ ( |[0-9])* Singapore [0-9]{6,6}"

biz_addr = "[0-9]{1,3}[a-zA-Z]? [a-zA-Z ]+ (.|[0-9])* Singapore [0-9]{6,6}"

#res_addr = "[Bb]lock [0-9]{1,4}[A-Z]? #[0-9]{1,2}\-[0-9]{2,4} " + "[A-Za-z ]+ ( |[0-9])* Singapore [0-9]{6,6}"

#biz_addr = "[0-9]{1,4} ([A-Za-z]+ )+(Avenue|Street|Road|Crescent|Drive|Way)"+ "( |[0-9])* Singapore [0-9]{6,6}"

addr = "("+res_addr+")|("+biz_addr+")"

p = re.compile(addr)

check_match(p.search("Block 299C #12-456 Ang Mo Kio Street 21 Singapore 543299"))

check_match(p.search("block 2C #11-26 King Street 21 Singapore 123499"))

check_match(p.search("180 Ang Mo Kio Avenue 8 Singapore 569830"))

check_match(p.search("180 Ang Mo Kio Road Singapore 569830"))

Block 299C #12-456 Ang Mo Kio Street 21 Singapore 543299

block 2C #11-26 King Street 21 Singapore 123499

180 Ang Mo Kio Avenue 8 Singapore 569830

180 Ang Mo Kio Road Singapore 569830

Google Sites

Report abuse