PRACTICAL GUIDE
Regular Expressions for Everyday Tasks
The dozen pieces of regex syntax that cover almost every real job, the traps that cost hours, and a discipline for testing patterns before you trust them.
Last updated
What regex is genuinely good at
Finding and replacing patterns in text: extracting all the dates from a log, renaming a variable across a file, splitting a messy column, validating a reference number against a known format. For these it is unmatched, and a few minutes learning it repays itself many times over.
What it is bad at is anything with nested structure. Parsing HTML, JSON or source code with a regular expression is the classic mistake — those formats can nest arbitrarily deep, and no pattern can track that. Use a parser.
The pieces that cover most of the work
Character classes: \d a digit, \w a letter, digit or underscore, \s whitespace, and a full stop for any character. Their capitalised forms mean the opposite — \D is any non-digit.
Quantifiers: * for zero or more, + for one or more, ? for optional, and {2,4} for a specific range.
Anchors: ^ for the start, $ for the end, and \b for a word boundary, which is what stops a search for cat from matching inside category.
Groups and alternation: parentheses to capture a piece for reuse, and a vertical bar for or. Square brackets define your own set, and a leading caret inside them negates it.
Greedy versus lazy is the classic trap
Quantifiers grab as much as they can by default. Applied to a line of HTML, a pattern of angle bracket, dot star, angle bracket matches from the first tag to the last one on the line, not the first tag alone. Adding a question mark after the quantifier makes it lazy, so it stops at the earliest possible match. Nearly every bewildering over-match traces back to this.
Escaping, and why patterns look like noise
Characters such as the full stop, plus, asterisk, question mark, parentheses, square brackets and backslash have special meanings, so matching them literally means escaping each with a backslash. If your pattern lives inside a string in code, that string may need its own escaping on top, which is how you end up with four backslashes to match one. Where a language offers raw strings or literal regex syntax, use it.
Do not write an email validator
The regular expression that fully implements the email address specification is famously several thousand characters long, and the short ones circulating online reject perfectly valid addresses — plus signs, new top-level domains, non-Latin characters. Check for an at sign with something plausible either side, then verify by sending a confirmation message. Delivery is the only real proof, and the same logic applies to phone numbers, postcodes and names.
Test against the awkward cases first
A pattern that works on your three example lines proves very little. Test it against the empty string, the value with unexpected whitespace, the one with an accented character, the duplicate, and the line that is nearly but not quite right — that last one is what tells you whether the pattern is precise or merely permissive.
Building the pattern incrementally in a tester, watching what each addition does to the matches, is far faster than writing the whole thing and debugging it inside an application.
Flags and dialects
The common flags are g to find every match rather than the first, i to ignore case, and m to make the anchors apply per line. Beyond that, dialects differ: JavaScript, Python, PCRE, and the grep family all have their own extensions and gaps. A pattern copied from an answer written for another language may need adjusting, and lookbehind in particular has patchy support.
Frequently asked questions
Why does my pattern match more than I expected?
Almost always a greedy quantifier. Make it lazy by adding a question mark after it, or replace the dot with a more specific character class that cannot cross the boundary you care about.
Can regex parse HTML or JSON?
Not reliably. Both allow arbitrary nesting, and regular expressions cannot count nesting depth. For a one-off extraction from a file you control it may be good enough; in production code it will break on the first document that is shaped slightly differently.
Is a complicated regex a performance risk?
It can be. Certain patterns with nested quantifiers cause catastrophic backtracking, where matching time explodes exponentially on an input designed to trigger it — a real denial-of-service vector when the pattern runs against user input. Keep patterns simple and avoid nesting one quantifier inside another.
Should I add comments to a regex?
Yes, where the language allows it. Several dialects have an extended mode that ignores whitespace and permits comments inside the pattern, which turns an unreadable line into something maintainable. Where that is unavailable, a comment above the pattern explaining what it matches is the minimum kindness to the next reader.