FilterHN

Regular expressions that work "everywhere"

49 points

by ColinWright

2 days ago

| past

| 14 comments

| johndcook.com

| HN

▲

tonyg

6 minutes ago

[-]

That's one of the reasons RFC 9485, "I-Regexp: An Interoperable Regular Expression Format", is important.

https://datatracker.ietf.org/doc/html/rfc9485

▲

JdeBP

6 hours ago

[-]

The author is circling around, but not quite reaching, a statement that POSIX Basic Regular Expressions work everywhere, with the caveat that that not everyone has caught up with version 8 of the Single Unix Specification, which has slightly changed BREs.

▲

rtpg

2 hours ago

[-]

Emacs in particular I suffer so much from basically guessing what needs to be escaped or not. I know `rx` exists[0] as an alternative but it's not really fun to use.

Even beyond the regex syntax itself, you often also start running into encoding problems when trying to actually use them. Typing the regex in a shell? Make sure to esacpe stuff properly. Regex in Python? Make sure it's a raw string. Etc etc etc

It's a modern miracle we're at least within rhyming distance of how to write regexes in most tools.

[0]: https://www.gnu.org/software/emacs/manual/html_node/elisp/Rx...

▲

agnishom

6 hours ago

[-]

A while ago, we wrote a paper about finding regexes which match the same way in both the greedy semantics and the leftmost maximal semantics.

https://par.nsf.gov/servlets/purl/10534654

▲

codetiger

2 hours ago

[-]

I built my Rust library for JSONLogic and use bindings for other languages after similar frustrations with Rule engines, template engines and IFTTT engines. https://github.com/GoPlasmatic/datalogic-rs

▲

MathMonkeyMan

7 hours ago

[-]

I've always been a stickler for being specific about which regex language your thing accepts, and whether it is to match any substring, or a prefix, or a suffix, or the whole thing, or a line, or a substring of a line, or whatever.

Here are some of the [more popular][1] ones, and then there are PCRE and Python.

It took me a while to learn that some of the older ones you see in e.g. grep are [specified by POSIX][2].

[1]: https://cppreference.com/cpp/regex#Regular_expression_gramma...

[2]: https://pubs.opengroup.org/onlinepubs/009696899/basedefs/xbd...

▲

myroon5

4 hours ago

[-]

JSON schema's docs also have a recommended regular expression subset:

https://json-schema.org/understanding-json-schema/reference/...

▲

galaxyLogic

1 hour ago

[-]

2 RegExp problems:

1. You can not compose a bigger regexp out of smaller ones

2. A regexp can not "call" other regexps

▲

wwind123

1 hour ago

[-]

To do regex matching efficiently, you need to compile the pattern before using it. That'd exclude dynamically "calling" other regex patterns. But bigger regex pattern strings can be composed from smaller regex pattern strings. You'd just need to do the composition before the compilation.

▲

ystlum

17 minutes ago

[-]

Also define blocks if all someone wants is to break the pattern up to make it more readable.

▲

pmarreck

3 hours ago

[-]

I've become a fan of whatever PCRE2 understands

▲

ok_dad

3 hours ago

[-]

Go stdlib regexp package does not support back references, as it uses the RE2 engine. You can use them in replace but not matching.

▲

masklinn

2 hours ago

[-]

Regexp does not use re2, it is a separate implementation of the same concepts.

▲

quotemstr

5 hours ago

[-]

It drives me nuts when a developer documents something or other as being a "regex" but doesn't mention which dialect of regulation expression he's talking about. This habit is particularly common in the Rust, JavaScript, and Python communities, which seem to forget that their language's regular expression language isn't universal.

▲

zahlman

5 hours ago

[-]

Why? Of course it means the dialect that is most directly supported by that language (by builtins or the standard library). And why should they have to consider other dialects? They aren't reading regexes from user input (or they'd be a lot more concerned about sanitization, catastrophic backtracking etc.), and their fellow developers all grok the conventions.

▲

bartread

4 hours ago

[-]

I’d imagine precisely because they might be collecting regexes from user input such as parameter values or search terms, and the user may not know or care which technology your tool or service is built with. However, they will need to know which regex dialect(s) you support.

And I’d further bet that people who are casual about specifying that are relatively strongly correlated with people who are casual about santization, catastrophic backtracking, etc. (At least based on code I’ve seen over the decades.)

▲

quotemstr

2 hours ago

[-]

Because I don't know what language your program is even written in! Why should I know or care that you chose, e.g. TypeScript, when I'm trying to use or configure your program and don't know how to spell this or that regex concept?

▲

xigoi

1 hour ago

[-]

Same applies to “Markdown”.

▲

LoganDark

6 hours ago

[-]

> the special characters . * ^ $

These already do not work in many tools which require those special characters to be escaped to have any meaning. An easy example is GNU grep, sed, etc. which use BRE ("Basic Regular Expressions") by default. The article mentions GNU coreutils but does not explain that `-E` is required to fix that behavior.

▲

jonstewart

4 hours ago

[-]

Then there’s not just the issue of whether the engine supports a particular syntactical feature but the issue of matching semantics. Perl/PCRE’s semantics are far different from POSIX’s and some implementations different semantics altogether (and quite reasonably).

▲

Resonix

2 days ago

[-]

why I built this

▲

greazy

6 hours ago

[-]

I think you forgot to post a link?

▲

codetiger

2 hours ago

[-]

I think he meant the post title. As the author shows his work finally in the article.