RE: How to grok a complicated regex?

[Drew Adams <drew.adams@oracle.com>]

> I’m not talking about changing the representation, but about making the
> existing one (which I agree is not /that/ bad) more comprehensible.
> Font lock, grouping and unescaping backslashes would be definitely helpful.
> 
> OTOH, I can imagine that some kind of diagrams might be helpful for
> someone.  The point is, in the end you have to read/write these regexen
> in their normal form anyway, so why not train yourself to understand
> their “default” representation instead of adding the burden of
> translationg between representations?

I agree that a visual aid can help with learning - about regexps
in general and about Emacs regexp syntax in particular.

The Emacs Wiki page about regexps provides suggestions about learning
regexp syntax: http://www.emacswiki.org/emacs/RegularExpression.

Incremental regexp searching (`C-M-s') is one good tool for learning.
What it does not help so much with is subgroup matching - keeping
the different groups straight when there are several possibilities.

Rasmus mentioned that `visual-regexp.el' can help with that. Likewise,
Icicles search: it highlights different subgroup matches differently.

Here is a screenshot that shows a complex regexp (5 groups) and a
diagram that maps each group to its highlighting:
http://www.emacswiki.org/emacs/RegularExpression#RegexpsInIcicles

The regexp: "(\([-a-z*]+\) *\((\(([-a-z]+ *\([^)]*\))\))\).*".
A left paren, a name, possibly some whitespace, two left parens,
a name, possibly some whitespace, possibly non right-paren chars,
two right parens, and any chars other than newline.  But grouped
in a particular way.

I find that it is more often the case, for a complicated regexp,
that you encounter it readymade (in some existing code), and you
want to see what it is all about and perhaps make a modification
to it.  That use case is more typical than is creating a complex
regexp from scratch.  As Emanuel said, such regexps are often
arrived at incrementally - they start simpler and evolve.

I recommend playing with existing regexps this way, seeing what
they match by using them with a visual tool such as Icicles search,
`visual-regexp.el', or even `C-M-s'.  A tour through the Emacs
source code will show you plenty of interesting regexps you can
play with - font-lock keywords and patterns defining Emacs pages,
sentences, etc.