ELisp: special read syntax for regular expressions?

ELisp: special read syntax for regular expressions?

[Ilya Zakharevich]

A few of years ago RMS asked me what one could do to make ELisp regular
expressions more handable.

Now, when the support for Perl REx in CPerl mode is finally finished,
I think I know the answer: one needs to allow a special read syntax
for regular expressions (the result being an ordinary string).  This
way one can avoid the acute backslashitis [by inverting \( and (,
etc], the regular expressions may be specially indented, and the
syntax highlighter would have a chance to remove most of the remaining
complexity by proper highlighting (similar to CPerl one).

What do you think?

Yours,
Ilya

P.S.  I put example of how my (Perl) test "code" is highlighted to

	ilyaz.org/software/tmp/CPerl-REx-highlight.png

Re: ELisp: special read syntax for regular expressions?

[David Hansen]

On Sat, 5 Apr 2008 11:17:10 +0000 (UTC) Ilya Zakharevich wrote:

> A few of years ago RMS asked me what one could do to make ELisp regular
> expressions more handable.
>
> Now, when the support for Perl REx in CPerl mode is finally finished,
> I think I know the answer: one needs to allow a special read syntax
> for regular expressions (the result being an ordinary string).  This
> way one can avoid the acute backslashitis [by inverting \( and (,
> etc], the regular expressions may be specially indented, and the
> syntax highlighter would have a chance to remove most of the remaining
> complexity by proper highlighting (similar to CPerl one).
>
> What do you think?

Have a look at the Gauche Scheme implementation.  It has a regexp read
syntax: #/.../.

Would be nice to have something like this in Emacs.  It can be hard to
keep track of all the backslashes.

David

Re: ELisp: special read syntax for regular expressions?

[Tim X]

Ilya Zakharevich <nospam-abuse@ilyaz.org> writes:

> A few of years ago RMS asked me what one could do to make ELisp regular
> expressions more handable.
>
> Now, when the support for Perl REx in CPerl mode is finally finished,
> I think I know the answer: one needs to allow a special read syntax
> for regular expressions (the result being an ordinary string).  This
> way one can avoid the acute backslashitis [by inverting \( and (,
> etc], the regular expressions may be specially indented, and the
> syntax highlighter would have a chance to remove most of the remaining
> complexity by proper highlighting (similar to CPerl one).
>
> What do you think?
>
> Yours,
> Ilya
>
> P.S.  I put example of how my (Perl) test "code" is highlighted to
>
> 	ilyaz.org/software/tmp/CPerl-REx-highlight.png

This sounds like a good idea. Many people, including myself, do find
elisp's REs a bit painful and its noticed even more these days since
perl compatible REs seem to have become a bit of a default standard. 

A modified read syntax certainly seems to be feasible and to my mind it
fits with a lisp way of doing things. Will be interesting to see how
easily that can be worked into emacs in a consistent way.

Tim


-- 
tcross (at) rapttech dot com dot au

Re: ELisp: special read syntax for regular expressions?

[David Combs]

Perl-6 has some totally different syntax for REs.

Not that I know what it is -- but does that look reasonable
for this use?

David

Re: ELisp: special read syntax for regular expressions?

[Peter Dyballa]

Am 05.04.2008 um 13:17 schrieb Ilya Zakharevich:
> What do you think?


The idea is good. The sample looks terrible. Too much, too intense  
colours – where is the RegExp? Using a shy colour for [:<type>:] is a  
good choice: this syntax is too easy.

Would it work and be sensible to keep an unfinished expression, for  
example "[-_a-dg", in a reddish colour that then changes to something  
regular when correctly finished?

--
Greetings

   Pete

It's not the valleys in life I dread so much as the dips.
				– Garfield

Re: ELisp: special read syntax for regular expressions?

[Ilya Zakharevich]

[A complimentary Cc of this posting was sent to
Peter Dyballa 
<Peter_Dyballa@Web.DE>], who wrote in article <mailman.10068.1207474538.18990.help-gnu-emacs@gnu.org>:
> The idea is good. The sample looks terrible.

So are the RExes.  ;-)

> Too much,

CPerl/Microdocs/Faces menu entry explaines which faces are used for
what (or see `cperl-tips-faces').  In your judgement, which constructs
should be "glued" together into one face?

> too intense colours.  Using a shy colour for [:<type>:] is a =20=
> good choice: this syntax is too easy.

Thanks, but no thanks.  Please keep your comments which colors look
good for you for your personal consumption (and/or your customization
file).

(The reason why variable-name is used for POSIX classes is simply that
all the other faces are already used for different purposes. ;-] ;-[)

>  =96 where is the RegExp

Do not know what you mean here.

> Would it work and be sensible to keep an unfinished expression, for =20
> example "[-_a-dg", in a reddish colour that then changes to something =20=
> 
> regular when correctly finished?

The constructs found unparsable are shown as such (see \q in the example).

Yours,
Ilya

Re: ELisp: special read syntax for regular expressions?

[Peter Dyballa]

Am 06.04.2008 um 23:38 schrieb Ilya Zakharevich:

>> Too much,
>
> CPerl/Microdocs/Faces menu entry explaines which faces are used for
> what (or see `cperl-tips-faces').  In your judgement, which constructs
> should be "glued" together into one face?

Separators (high-lighted? bold face?), parentheses used semantically  
(not as element to be matched or found), "modifiers" like s or m  
before and [egimosx] after the RegExp, referenced text (i.n., inside  
"(" and ")"), back-references, class specifications ("[:alpha:]") if  
not already referenced, other text.

It might be desirable to use in the from part darker shades of the  
colour than in the to part, i.e., <colour>3 in from and <colour>2 in  
to section. Since GNU Emacs has a very good undo function it might be  
better to emphasise on the from part, to think better about this, to  
achieve on first try the desired change. (At least I have to try more  
than once from time to time.)

>
>> too intense colours.  Using a shy colour for [:<type>:] is a =20=
>> good choice: this syntax is too easy.
>
> Thanks, but no thanks.  Please keep your comments which colors look
> good for you for your personal consumption (and/or your customization
> file).

There is good evidence found that different colours have different  
effect on humans. Blueish and greenish colours have a relaxing effect  
on the eye when used as a background colour on the computer screen  
(more exert muscle activity is needed to perceive text in reddish  
colour sharply). This also tells that text in blueish and greenish  
colour is easier sharply perceived, i.e., more visible, lighter to  
read as text with a certain meaning. Reddish colour does not seem to  
need text to become evident as a warning. (At least I do not read red  
text so often, for me it's OK that there is some, standing for "an  
error just happened!")

>
>>  =96 where is the RegExp
>
> Do not know what you mean here.

There are so many and different colours in use that I hardly can  
recognise any REs. I have the feeling that most lines of your  
examples are faulty, but I confess that I do not know all Perl  
RegExps by heart.

--
Greetings

   Pete

Imbecility, n.:
	A kind of divine inspiration, or sacred fire affecting
	censorious critics of this dictionary.
			– Ambrose Bierce: _The Devil's Dictionary_

Re: ELisp: special read syntax for regular expressions?

[Ilya Zakharevich]

[A complimentary Cc of this posting was sent to
Peter Dyballa 
<Peter_Dyballa@Web.DE>], who wrote in article <mailman.10316.1208041979.18990.help-gnu-emacs@gnu.org>:

> >> Too much,

> > CPerl/Microdocs/Faces menu entry explaines which faces are used for
> > what (or see `cperl-tips-faces').  In your judgement, which constructs
> > should be "glued" together into one face?

> Separators (high-lighted? bold face?), parentheses used semantically =20
> (not as element to be matched or found), "modifiers" like s or m =20
> before and [egimosx] after the RegExp, referenced text (i.n., inside =20
> "(" and ")"), back-references, class specifications ("[:alpha:]") if =20
> not already referenced, other text.

I have no idea what you are talking about.  Currently different parts
of a REx are merged into 10 distinct groups.  Apparently, your "too
much" comment suggests that you want some of these groups merged
together.

Do not see anything like this in your reply.  Could you use complete
sentences, please?  (For convenience of other readers, I reproduce the
current doc-strings below.)

> It might be desirable to use in the from part darker shades of the =20
> colour than in the to part, i.e., <colour>3 in from and <colour>2 in =20
> to section.

Do not know what you are talking about.  Do you mean the REx part and
the substitution part?  The substitution part is just a string...

> Since GNU Emacs has a very good undo

Could not agree less...

> function it might be =20=
> 
> better to emphasise on the from part, to think better about this, to =20
> achieve on first try the desired change. (At least I have to try more =20=
> 
> than once from time to time.)

Have no clue what it is you discuss now...

> >>  =3D96 where is the RegExp
> >
> > Do not know what you mean here.
> 
> There are so many and different colours in use that I hardly can =20
> recognise any REs. I have the feeling that most lines of your =20
> examples are faulty, but I confess that I do not know all Perl =20
> RegExps by heart.

This is what the fontification is for.  There is one error; it is
highlighted...  The other stuff is highlighted in such a way that one
can always see where any escape starts and ends, so one has a better
chance to find it in the docs.

Hope this helps,
Ilya

=======================================================

In regular expressions (including character classes):
  `font-lock-string-face'	\"Normal\" stuff and non-0-length constructs
  `font-lock-constant-face':	Delimiters
  `font-lock-warning-face'	Special-cased m// and s//foo/,
				Mismatched closing delimiters, parens
				we couldn't match, misplaced quantifiers,
				unrecognized escape sequences
  `cperl-nonoverridable-face'	Modifiers, as gism in m/REx/gism
  `font-lock-type-face'		escape sequences with arguments (\\x \\23 \\p \\N)
				and others match-a-char escape sequences
  `font-lock-keyword-face'	Capturing parens, and |
  `font-lock-function-name-face' Special symbols: $ ^ . [ ] [^ ] (?{ }) (??{ })
				\"Range -\" in character classes
  `font-lock-builtin-face'	\"Remaining\" 0-length constructs, multipliers
				?+*{}, not-capturing parens, leading
				backslashes of escape sequences
  `font-lock-variable-name-face' Interpolated constructs, embedded code,
				POSIX classes (inside charclasses)
  `font-lock-comment-face'	Embedded comments

Re: ELisp: special read syntax for regular expressions?

[Ilya Zakharevich]

[A complimentary Cc of this posting was NOT [per weedlist] sent to
Ilya Zakharevich 
<nospam-abuse@ilyaz.org>], who wrote in article <ftrgtm$eaj$1@agate.berkeley.edu>:
> > It might be desirable to use in the from part darker shades of the =20
> > colour than in the to part, i.e., <colour>3 in from and <colour>2 in =20
> > to section.

> Do not know what you are talking about.  Do you mean the REx part and
> the substitution part?  The substitution part is just a string...

Oups: a string, or CODE (depending on presence of ///e); highlighted
accordingly.

Sorry,
Ilya

Re: ELisp: special read syntax for regular expressions?

[Mike Mattie]

[-- Attachment #1: Type: text/plain, Size: 932 bytes --]

On Sat, 5 Apr 2008 11:17:10 +0000 (UTC)
Ilya Zakharevich <nospam-abuse@ilyaz.org> wrote:

> A few of years ago RMS asked me what one could do to make ELisp
> regular expressions more handable.
> 
> Now, when the support for Perl REx in CPerl mode is finally finished,
> I think I know the answer: one needs to allow a special read syntax
> for regular expressions (the result being an ordinary string).  This
> way one can avoid the acute backslashitis [by inverting \( and (,
> etc], the regular expressions may be specially indented, and the
> syntax highlighter would have a chance to remove most of the remaining
> complexity by proper highlighting (similar to CPerl one).
> 
> What do you think?

I would love it. Regex's don't need a slash tax to be cryptic.
 
> Yours,
> Ilya
> 
> P.S.  I put example of how my (Perl) test "code" is highlighted to
> 
> 	ilyaz.org/software/tmp/CPerl-REx-highlight.png

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: ELisp: special read syntax for regular expressions?

[Ilya Zakharevich]

[A complimentary Cc of this posting was sent to
David Hansen 
<david.hansen@gmx.net>], who wrote in article <mailman.10052.1207428807.18990.help-gnu-emacs@gnu.org>:
> > I think I know the answer: one needs to allow a special read syntax
> > for regular expressions (the result being an ordinary string).  This
> > way one can avoid the acute backslashitis [by inverting \( and (,
> > etc], the regular expressions may be specially indented, and the
> > syntax highlighter would have a chance to remove most of the remaining
> > complexity by proper highlighting (similar to CPerl one).
> >
> > What do you think?
> 
> Have a look at the Gauche Scheme implementation.  It has a regexp read
> syntax: #/.../.

IMO, having SLASH hardwired as a delimiter is a silly choice; it makes
matching a slash harder, UNNECESSARILY.  If one MUST hardwire some
delimiters, it should better be () (since un-backwacked parens are
almost guarantied to be matched inside a regular expression, one needs
no extra backwacking to enclose the REx into MATCHED parens).

Any thoughts?  Is /(REX) prohibited by some considerations?

Yours,
Ilya

P.S.  I just released v6.2 of CPerl mode; now it has a menu entry
      Perl/Micro-docs/Show-Faces which inserts an example Perl code
      with (all?) possible faces shown (when proper facification is
      enabled).  This may simplify the part of this thread concerned
      with REx facification in CPerl.

      See ilyaz.org/software/emacs

Re: ELisp: special read syntax for regular expressions?

[Joost Diepenmaat]

Ilya Zakharevich <nospam-abuse@ilyaz.org> writes:

> [A complimentary Cc of this posting was sent to
> David Hansen 
> <david.hansen@gmx.net>], who wrote in article <mailman.10052.1207428807.18990.help-gnu-emacs@gnu.org>:
>> Have a look at the Gauche Scheme implementation.  It has a regexp read
>> syntax: #/.../.
>
> IMO, having SLASH hardwired as a delimiter is a silly choice; it makes
> matching a slash harder, UNNECESSARILY.  If one MUST hardwire some
> delimiters, it should better be () (since un-backwacked parens are
> almost guarantied to be matched inside a regular expression, one needs
> no extra backwacking to enclose the REx into MATCHED parens).

Agreed. From using loads of regexes in perl, I much prefer to use qr(
.. ) or qr{ ... }. Or at least, forward slashes in matches are just much
too common in many typical matches (URLs & paths, for instance). But you
would know that :-)

> Any thoughts?  Is /(REX) prohibited by some considerations?

Dunno. I'd think that #{ ... } or #( ... ) or possibly even #r{ .. } or
#r( .. ) would be clearer as "special reader syntax". But I really don't
know enough about (emacs) lisp reader constructs.

> P.S.  I just released v6.2 of CPerl mode; now it has a menu entry
>       Perl/Micro-docs/Show-Faces which inserts an example Perl code
>       with (all?) possible faces shown (when proper facification is
>       enabled).  This may simplify the part of this thread concerned
>       with REx facification in CPerl.
>
>       See ilyaz.org/software/emacs

That link doesn't seem to exist; it redirects to
http://www.yeah.com/software/emacs

Cheers,
Joost.

-- 
Joost Diepenmaat | blog: http://joost.zeekat.nl/ | work: http://zeekat.nl/

Re: ELisp: special read syntax for regular expressions?

[Ilya Zakharevich]

[A complimentary Cc of this posting was sent to
Joost Diepenmaat 
<joost@zeekat.nl>], who wrote in article <87tzi4uhqw.fsf@zeekat.nl>:
> > P.S.  I just released v6.2 of CPerl mode; now it has a menu entry
> >       Perl/Micro-docs/Show-Faces which inserts an example Perl code
> >       with (all?) possible faces shown (when proper facification is
> >       enabled).  This may simplify the part of this thread concerned
> >       with REx facification in CPerl.
> >
> >       See ilyaz.org/software/emacs
> 
> That link doesn't seem to exist; it redirects to
> http://www.yeah.com/software/emacs

It works here...

Yours,
Ilya

Re: ELisp: special read syntax for regular expressions?

[David Hansen]

On Mon, 14 Apr 2008 23:27:53 +0000 (UTC) Ilya Zakharevich wrote:

> [A complimentary Cc of this posting was sent to
> David Hansen 
> <david.hansen@gmx.net>], who wrote in article <mailman.10052.1207428807.18990.help-gnu-emacs@gnu.org>:
>> > I think I know the answer: one needs to allow a special read syntax
>> > for regular expressions (the result being an ordinary string).  This
>> > way one can avoid the acute backslashitis [by inverting \( and (,
>> > etc], the regular expressions may be specially indented, and the
>> > syntax highlighter would have a chance to remove most of the remaining
>> > complexity by proper highlighting (similar to CPerl one).
>> >
>> > What do you think?
>> 
>> Have a look at the Gauche Scheme implementation.  It has a regexp read
>> syntax: #/.../.
>
> IMO, having SLASH hardwired as a delimiter is a silly choice; it makes
> matching a slash harder, UNNECESSARILY.  If one MUST hardwire some
> delimiters, it should better be () (since un-backwacked parens are
> almost guarantied to be matched inside a regular expression, one needs
> no extra backwacking to enclose the REx into MATCHED parens).
>
> Any thoughts?  Is /(REX) prohibited by some considerations?

The lisper will expect a `#' to see that it it's a special read syntax,
and it's probably a good idea to stick with CL conventions/standards[1].

I'd vote for #/<any paren>REX<matching closing>.  But I have to admit
that I don't know anything about the emacs lisp reader.  Maybe the
discussion should be moved to emacs-devel.

David

Footnotes: 
[1]  http://www.lispworks.com/documentation/HyperSpec/Body/02_dh.htm