Low level trickery for changing character syntax?

Low level trickery for changing character syntax?

[Thorsten Jolitz]

Hi List, 

assume an imaginary elisp library gro.el I cannot (or don't want to)
change that is used on files of type A, with functions matching these
kinds of strings:

#+begin_src emacs-lisp
  (defconst rgxp-1 "^[*] [*]Fat[*]$")

  (defun foo (strg)
    (and (string-match "^\\*+[ \t]* \\*.+\\*" strg)
         (string-match rgxp-1 strg)))
#+end_src

#+results:
: foo

#+begin_src emacs-lisp
(foo "* *Fat*")
#+end_src

#+results:
: 0

#+begin_src emacs-lisp
(foo "+ *Fat*")
#+end_src

#+results:

Now assume I want to use gro.el functionality on files of type B
such that it matches strings likes this:

#+begin_src emacs-lisp
(foo "// # *Fat*//" )
#+end_src

In short, when called from file.type-A, I want foo to match "// #
*Fat*//", while it should only match "* *Fat*" when called from
file.type-B (without changing foo or rgxp-1).
 
Thus in rgxp-1 and in foo, "^" would need to be replaced with "^// ",
the first "*" would need to be replaced with "#" (the other occurences
not), and "$" would need to be replaced with "//$".

Now I wonder what would be the best way (or at least a possible way) to
achieve this with Emacs low-level trickery (almost) without touching
gro.el. I don't enough know about syntax table low-level stuff besides
reading the manual, so these are only vague speculations:

 1. Change the syntax-table of gro.el whenever it is applied to files of
 type B such that "^" is seen as "^// ", "*" as "#" etc.?

 2. Define new categories and put "^" "*" and "$" in them, and somehow
 load/activate these categories conditional on the type of file gro.el
 functionality is called upon. These categories should then achieve that
 "^" is seen as "^// " etc when the categories are loaded?

 3. Define "^" and "$", when found at beg/end of a string, as 'generic
 comment delimiter, and define "/" as generic comment delimiter too, such
 that "^//" and "//$" are matched by "^" and "$"?

I know that these ideas do not and cannot work as described, but I'm
looking for a hint which idea could possibly work? What would be the way
to go? 

Or is this completely unrealistic and the only way to achieve it is to
change the hardcoded regexps in (imaginary) library gro.el?

-- 
cheers,
Thorsten

Re: Low level trickery for changing character syntax?

[Thorsten Jolitz]

Thorsten Jolitz <tjolitz@gmail.com> writes:

> In short, when called from file.type-A, I want foo to match "// #
> *Fat*//", while it should only match "* *Fat*" when called from
> file.type-B (without changing foo or rgxp-1).

sorry, vice versa of course

 - file.type-A: match "* *Fat*"
 - file.type-B: match "// # *Fat*//"

I became confused (although it is really irrelevant for the example).

-- 
cheers,
Thorsten

Re: Low level trickery for changing character syntax?

[Andreas Röhler]

Am 08.04.2014 19:00, schrieb Thorsten Jolitz:
>
> Hi List,
>
> assume an imaginary elisp library gro.el I cannot (or don't want to)
> change that is used on files of type A, with functions matching these
> kinds of strings:
>
> #+begin_src emacs-lisp
>    (defconst rgxp-1 "^[*] [*]Fat[*]$")
>
>    (defun foo (strg)
>      (and (string-match "^\\*+[ \t]* \\*.+\\*" strg)
>           (string-match rgxp-1 strg)))
> #+end_src
>
> #+results:
> : foo
>
> #+begin_src emacs-lisp
> (foo "* *Fat*")
> #+end_src
>
> #+results:
> : 0
>
> #+begin_src emacs-lisp
> (foo "+ *Fat*")
> #+end_src
>
> #+results:
>
> Now assume I want to use gro.el functionality on files of type B
> such that it matches strings likes this:
>
> #+begin_src emacs-lisp
> (foo "// # *Fat*//" )
> #+end_src
>
> In short, when called from file.type-A, I want foo to match "// #
> *Fat*//", while it should only match "* *Fat*" when called from
> file.type-B (without changing foo or rgxp-1).
>
> Thus in rgxp-1 and in foo, "^" would need to be replaced with "^// ",
> the first "*" would need to be replaced with "#" (the other occurences
> not), and "$" would need to be replaced with "//$".
>
> Now I wonder what would be the best way (or at least a possible way) to
> achieve this with Emacs low-level trickery (almost) without touching
> gro.el. I don't enough know about syntax table low-level stuff besides
> reading the manual, so these are only vague speculations:
>
>   1. Change the syntax-table of gro.el whenever it is applied to files of
>   type B such that "^" is seen as "^// ", "*" as "#" etc.?
>
>   2. Define new categories and put "^" "*" and "$" in them, and somehow
>   load/activate these categories conditional on the type of file gro.el
>   functionality is called upon. These categories should then achieve that
>   "^" is seen as "^// " etc when the categories are loaded?
>
>   3. Define "^" and "$", when found at beg/end of a string, as 'generic
>   comment delimiter, and define "/" as generic comment delimiter too, such
>   that "^//" and "//$" are matched by "^" and "$"?
>
> I know that these ideas do not and cannot work as described, but I'm
> looking for a hint which idea could possibly work? What would be the way
> to go?
>
> Or is this completely unrealistic and the only way to achieve it is to
> change the hardcoded regexps in (imaginary) library gro.el?
>

Possible is a lot, the easiest IMO would be switching the regexp

if A-type:

...

if B-type:

Re: Low level trickery for changing character syntax?

[Thorsten Jolitz]

Andreas Röhler <andreas.roehler@easy-emacs.de> writes:

> Am 08.04.2014 19:00, schrieb Thorsten Jolitz:
>>
>> Hi List,
>>
>> assume an imaginary elisp library gro.el I cannot (or don't want to)
>> change that is used on files of type A, with functions matching these
>> kinds of strings:
>>
>> #+begin_src emacs-lisp
>>    (defconst rgxp-1 "^[*] [*]Fat[*]$")
>>
>>    (defun foo (strg)
>>      (and (string-match "^\\*+[ \t]* \\*.+\\*" strg)
>>           (string-match rgxp-1 strg)))
>> #+end_src
>>
>> #+results:
>> : foo
>>
>> #+begin_src emacs-lisp
>> (foo "* *Fat*")
>> #+end_src
>>
>> #+results:
>> : 0
>>
>> #+begin_src emacs-lisp
>> (foo "+ *Fat*")
>> #+end_src
>>
>> #+results:
>>
>> Now assume I want to use gro.el functionality on files of type B
>> such that it matches strings likes this:
>>
>> #+begin_src emacs-lisp
>> (foo "// # *Fat*//" )
>> #+end_src
>>
>> In short, when called from file.type-A, I want foo to match "// #
>> *Fat*//", while it should only match "* *Fat*" when called from
>> file.type-B (without changing foo or rgxp-1).
>>
>> Thus in rgxp-1 and in foo, "^" would need to be replaced with "^// ",
>> the first "*" would need to be replaced with "#" (the other occurences
>> not), and "$" would need to be replaced with "//$".
>>
>> Now I wonder what would be the best way (or at least a possible way) to
>> achieve this with Emacs low-level trickery (almost) without touching
>> gro.el. I don't enough know about syntax table low-level stuff besides
>> reading the manual, so these are only vague speculations:
>>
>>   1. Change the syntax-table of gro.el whenever it is applied to files of
>>   type B such that "^" is seen as "^// ", "*" as "#" etc.?
>>
>>   2. Define new categories and put "^" "*" and "$" in them, and somehow
>>   load/activate these categories conditional on the type of file gro.el
>>   functionality is called upon. These categories should then achieve that
>>   "^" is seen as "^// " etc when the categories are loaded?
>>
>>   3. Define "^" and "$", when found at beg/end of a string, as 'generic
>>   comment delimiter, and define "/" as generic comment delimiter too, such
>>   that "^//" and "//$" are matched by "^" and "$"?
>>
>> I know that these ideas do not and cannot work as described, but I'm
>> looking for a hint which idea could possibly work? What would be the way
>> to go?
>>
>> Or is this completely unrealistic and the only way to achieve it is to
>> change the hardcoded regexps in (imaginary) library gro.el?
>>
>
> Possible is a lot, the easiest IMO would be switching the regexp
>
> if A-type:
>
> ...
>
> if B-type:

The question assumes the restriction that the functions of gro.el, that
work for files of type-A, should be used with files of type-B, but I
cannot change (the hundreds of hardcoded regexp strings scattered all
over the giant file) gro.el. 

When it becomes clear that nothing can be done under this restriction, I
start thinking about how to circumvent it, but for now I'm interested in
some fancy trickery at the syntax-table level - do not touch the
hard-coded regexp strings, but give a few characters a different meaning
when called from a file of type-B.

-- 
cheers,
Thorsten

Re: Low level trickery for changing character syntax?

[Andreas Röhler]

Am 08.04.2014 19:00, schrieb Thorsten Jolitz:
>
> Hi List,
>
> assume an imaginary elisp library gro.el I cannot (or don't want to)
> change that is used on files of type A, with functions matching these
> kinds of strings:
>
> #+begin_src emacs-lisp
>    (defconst rgxp-1 "^[*] [*]Fat[*]$")
>
>    (defun foo (strg)
>      (and (string-match "^\\*+[ \t]* \\*.+\\*" strg)
>           (string-match rgxp-1 strg)))
> #+end_src
>
> #+results:
> : foo
>
> #+begin_src emacs-lisp
> (foo "* *Fat*")
> #+end_src
>
> #+results:
> : 0
>
> #+begin_src emacs-lisp
> (foo "+ *Fat*")
> #+end_src
>
> #+results:
>
> Now assume I want to use gro.el functionality on files of type B
> such that it matches strings likes this:
>
> #+begin_src emacs-lisp
> (foo "// # *Fat*//" )
> #+end_src
>
> In short, when called from file.type-A, I want foo to match "// #
> *Fat*//", while it should only match "* *Fat*" when called from
> file.type-B (without changing foo or rgxp-1).
>
> Thus in rgxp-1 and in foo, "^" would need to be replaced with "^// ",
> the first "*" would need to be replaced with "#" (the other occurences
> not), and "$" would need to be replaced with "//$".
>
> Now I wonder what would be the best way (or at least a possible way) to
> achieve this with Emacs low-level trickery (almost) without touching
> gro.el. I don't enough know about syntax table low-level stuff besides
> reading the manual, so these are only vague speculations:
>
>   1. Change the syntax-table of gro.el whenever it is applied to files of
>   type B such that "^" is seen as "^// ", "*" as "#" etc.?
>
>   2. Define new categories and put "^" "*" and "$" in them, and somehow
>   load/activate these categories conditional on the type of file gro.el
>   functionality is called upon. These categories should then achieve that
>   "^" is seen as "^// " etc when the categories are loaded?
>
>   3. Define "^" and "$", when found at beg/end of a string, as 'generic
>   comment delimiter, and define "/" as generic comment delimiter too, such
>   that "^//" and "//$" are matched by "^" and "$"?
>
> I know that these ideas do not and cannot work as described, but I'm
> looking for a hint which idea could possibly work? What would be the way
> to go?
>
> Or is this completely unrealistic and the only way to achieve it is to
> change the hardcoded regexps in (imaginary) library gro.el?
>

You could define different syntax-tables and than call functions

if type-A
(with-syntax-table type-A ...

Re: Low level trickery for changing character syntax?

[Tassilo Horn]

Thorsten Jolitz <tjolitz@gmail.com> writes:

Hi Thorsten,

> assume an imaginary elisp library gro.el I cannot (or don't want to)
> change that is used on files of type A, with functions matching these
> kinds of strings:
>
> [...]
>
> Or is this completely unrealistic and the only way to achieve it is to
> change the hardcoded regexps in (imaginary) library gro.el?

I don't know a good generic way to achieve what you want, but maybe you
can get pretty far in a concrete case anyway.

I assume you want to use parts of org-mode in programming mode buffers.
But which parts?  I only use its convenient cycling of outline headings,
and that's pretty easy to setup.  I can post the code if wanted.

Bye,
Tassilo

Re: Low level trickery for changing character syntax?

[Thorsten Jolitz]

Andreas Röhler <andreas.roehler@easy-emacs.de> writes:

> Am 08.04.2014 19:00, schrieb Thorsten Jolitz:
>>
>> Hi List,
>>
>> assume an imaginary elisp library gro.el I cannot (or don't want to)
>> change that is used on files of type A, with functions matching these
>> kinds of strings:
>>
>> #+begin_src emacs-lisp
>>    (defconst rgxp-1 "^[*] [*]Fat[*]$")
>>
>>    (defun foo (strg)
>>      (and (string-match "^\\*+[ \t]* \\*.+\\*" strg)
>>           (string-match rgxp-1 strg)))
>> #+end_src
>>
>> #+results:
>> : foo
>>
>> #+begin_src emacs-lisp
>> (foo "* *Fat*")
>> #+end_src
>>
>> #+results:
>> : 0
>>
>> #+begin_src emacs-lisp
>> (foo "+ *Fat*")
>> #+end_src
>>
>> #+results:
>>
>> Now assume I want to use gro.el functionality on files of type B
>> such that it matches strings likes this:
>>
>> #+begin_src emacs-lisp
>> (foo "// # *Fat*//" )
>> #+end_src
>>
>> In short, when called from file.type-A, I want foo to match "// #
>> *Fat*//", while it should only match "* *Fat*" when called from
>> file.type-B (without changing foo or rgxp-1).
>>
>> Thus in rgxp-1 and in foo, "^" would need to be replaced with "^// ",
>> the first "*" would need to be replaced with "#" (the other occurences
>> not), and "$" would need to be replaced with "//$".
>>
>> Now I wonder what would be the best way (or at least a possible way) to
>> achieve this with Emacs low-level trickery (almost) without touching
>> gro.el. I don't enough know about syntax table low-level stuff besides
>> reading the manual, so these are only vague speculations:
>>
>>   1. Change the syntax-table of gro.el whenever it is applied to files of
>>   type B such that "^" is seen as "^// ", "*" as "#" etc.?
>>
>>   2. Define new categories and put "^" "*" and "$" in them, and somehow
>>   load/activate these categories conditional on the type of file gro.el
>>   functionality is called upon. These categories should then achieve that
>>   "^" is seen as "^// " etc when the categories are loaded?
>>
>>   3. Define "^" and "$", when found at beg/end of a string, as 'generic
>>   comment delimiter, and define "/" as generic comment delimiter too, such
>>   that "^//" and "//$" are matched by "^" and "$"?
>>
>> I know that these ideas do not and cannot work as described, but I'm
>> looking for a hint which idea could possibly work? What would be the way
>> to go?
>>
>> Or is this completely unrealistic and the only way to achieve it is to
>> change the hardcoded regexps in (imaginary) library gro.el?
>>
>
> You could define different syntax-tables and than call functions
>
> if type-A
> (with-syntax-table type-A ...

That looks like a promising approach, but I never worked with
syntax-tables so I ask myself:

Is it possible to redefine characters "^", "$" and "*" in a syntax-table
in such a way that the same hardcoded regexp, e.g.

  ,------------------
  | "^[*] [*]Fat[*]$"
  `------------------

matches "* *Fat*" when called (with-syntax-table type-A ...), but
matches e.g. "// # *Fat*//" when called (with-syntax-table type-B ...)? 

* First approach

(from the elisp manual)
,---------------------------------------------------------------------
| A syntax descriptor is a Lisp string that describes the syntax class
| and other syntactic properties of a character. When you want to
| modify the syntax of a character, that is done by calling the
| function modify-syntax-entry and passing a syntax descriptor as one
| of its arguments (see Syntax Table Functions).
| 
| The first character in a syntax descriptor must be a syntax class
| designator character. The second character, if present, specifies a
| matching character (e.g., in Lisp, the matching character for '(' is
| ')'); a space specifies that there is no matching character. Then
| come characters specifying additional syntax properties (see Syntax
| Flags).
| 
| If no matching character or flags are needed, only one character
| (specifying the syntax class) is sufficient.
| 
| For example, the syntax descriptor for the character '*' in C mode
| is ". 23" (i.e., punctuation, matching character slot unused, second
| character of a comment-starter, first character of a comment-ender),
| and the entry for '/' is '. 14' (i.e., punctuation, matching
| character slot unused, first character of a comment-starter, second
| character of a comment-ender).
`---------------------------------------------------------------------

I can see how give e.g. "^" a different syntax class from this quote,
maybe make it a comment-starter, but I cannot see how to make it match
the combination of itself, two comment-starters and a space if and only
if it follows a \", i.e. how to make 

,------
| (looking-at "^")
`------

match e.g.

,-------
| "// "
`-------

at the beginning of a line when called (with-syntax-table type-B ...)?

* Second approach

(from the elisp manual)
,---------------------------------------------------------------------
| When the syntax table is not flexible enough to specify the syntax
| of a language, you can override the syntax table for specific
| character occurrences in the buffer, by applying a syntax-table text
| property. See Text Properties, for how to apply text properties.
`---------------------------------------------------------------------

where I find:
,-------------------------------------------------------------------
| Properties with Special Meanings
| 
| Here is a table of text property names that have special built-in
| meanings.
| 
| syntax-table
|     The syntax-table property overrides what the syntax table says
|     about this particular character. See Syntax Properties.
`-------------------------------------------------------------------

So I could assign "^" some special value for its special text property
'syntax-table, but w/o an example how to achieve my goal this way I'm a
bit lost here.

* Third approach

(from the elisp manual)
,--------------------------------------------------------------------
| Categories
| 
| Categories provide an alternate way of classifying characters
| syntactically. You can define several categories as needed, then
| independently assign each character to one or more categories.
| Unlike syntax classes, categories are not mutually exclusive; it is
| normal for one character to belong to several categories.
`--------------------------------------------------------------------

category-tables are buffer-local like syntax-tables, what is useful in
my case. Say I define category-table "B" buffer-local in buffers of
type-B files. But what then? Would I have to put "^", "/" (or more
generally 'comment-start') and " " in that category, such that a single

,------
| (looking-at "^")
`------

matches

,-------
| "// "
`-------

when called from a buffer with buffer-local category "B"? I cannot see
how this should work.

-- 
cheers,
Thorsten

Org Minor Mode (was Re: Low level trickery for changing character syntax?)

[Thorsten Jolitz]

Tassilo Horn <tsdh@gnu.org> writes:

Hi Tassilo, 

> Thorsten Jolitz <tjolitz@gmail.com> writes:
>
> Hi Thorsten,

> I assume you want to use parts of org-mode in programming mode buffers.
> But which parts?  I only use its convenient cycling of outline headings,
> and that's pretty easy to setup.  I can post the code if wanted.

Right, I'm very much convinced that the power of Org-mode could be
unleashed in a true org-minor-mode that works in the comment-sections of
programming major-modes. 

With outshine.el and outorg.el this does exist already, and works quite
well (and is really useful, I use it all the time). But outshine uses
outline-minor-mode as "backend/engine", thus to bring it closer to the
tru Org-mode power, Org-mode functionality would have to be
reimplemented - a true uphill battle. 

Org-mode has gone a long way since its beginnings as an extension to
outline, and almost any functionality one could imagine is already there
and implemented. 

The major "flaw" of Org-mode that inhibits its use as minor-mode is the
wide-spread use of hard-coded regexps, i.e. regexps of this form

,--------------
| "^\\*+ ... $"
`--------------

in many many variants all over the place. Those three elements "^"
(bol), "$" (eol) and "*" (star) are not portable, since in
comment-sections of programming major-modes the above should look
like:

PicoLisp  (comment-start repeated with padding)
,--------------
| "^## *+ ... $" 
`--------------

Elisp outshine-style (comment-start repeated with padding)
,--------------
| "^;; *+ ... $" 
`--------------

old-school Elisp (comment-start repeated no padding, different star)
,--------------
| "^;;;+ ... $" 
`--------------

(?) CSS (comment-start with padding, comment-end)
,--------------
| "^/[*] [*]+ ... \/[*]$" 
`--------------

I'm almost done writing a library [[https://github.com/tj64/drx][(drx.el)]] that allows writing
declarative dynamic regexps for elisp like this:

# default org-mode
#+begin_src emacs-lisp
 (drx " foo" t "+" t)
#+end_src

#+results:
: ^\*+ foo$

but

# old-school elisp 
#+begin_src emacs-lisp
  (let ((drx-BOL "^;;")
        (drx-STAR ";"))
    (drx " foo" t "+" t))
#+end_src

#+results:
: ^;;;+foo$

and

# css
#+begin_src emacs-lisp
    (let ((drx-BOL "^/\\* ")
          (drx-EOL "\\*/$ ")
          (drx-STAR "\\*"))
      (drx " foo" t "+" t))
#+end_src

#+results:
: ^/\* \*+ foo\*/$ 

(unfortunately I learned to late about the existance of amazing rx.el
that can do these things too I guess)

In Org-mode there are 

 - global regexp constants :: a few dozens, easy to convert
 - buffer local regexp constants :: a few dozens, easy to convert, can
      be extracted using the "orgstruct-trick"
 - regexp snippets in functions :: hundreds of (looking-at ...),
      (re-search-forward ...) etc. with hard-coded regexp-snippets
      using one of the three non-portable chars ("^", "$", "*"). 

Converting Org-mode sources would be possible (some 600+ changes to
dozens of files in /lisp), but not easy and quite risky. And it would
require a 'paradigm change' wrt to future use of regexps - no more
hard-coded strings (or only regexps that take into accounts the needs
of an org-minor-mode) but rather dynamically calculated regexps
written with drx.el or rx.el or so. I'm not the one to decide this and
I have strong doubts if such massive changes will be supported.

I'm currently working on [[https://github.com/tj64/omm][Org Minor Mode]], which can be seen as

 - a port of outshine from old backend outline to new backend org-mode
 - a merge of orgstruct with outshine plus new ideas

I think orgstruct-mode and outshine a both based on one core idea:

 - outshine :: don't let users specify outline-regexp and level, let
               the minor-mode calculate them from major-modes comment-syntax.
 - outstruct :: run org commands in a temporary environment with Org's
                buffer-local variables set.

Add to this a new idea

 - org-minor-mode :: make the full power of Org-mode available in
                     comment-sections of programming-modes by
                     introducing one abstraction step into Org-mode
                     regexps (e.g. use drx-BOL instead of hardcoded
                     "^", drx-EOL instead of hardcoded "$", and
                     drx-STAR instead of hardcoded "*").

There is not much code to write for implementing org-minor-mode, its
about merging orgstruct with outshine (I'm on my way in omm.el) and to
somehow get around this 'hardcoded regexp' problem.

The keymap of omm.el will look like this:

,----------------------------------------------------------
| [...]
|  (define-key map (kbd "C-c C-x p")
|    (lambda () (interactive) (omm-cmd 'org-set-property)))
|  (define-key map (kbd "C-c C-d")
|    (lambda () (interactive) (omm-cmd 'org-deadline)))
|  (define-key map (kbd "C-c C-x t")
| [...]
`----------------------------------------------------------

with `omm-cmd' taking care of 

 - setting EOL, BOL and STAR according to major-mode

 - establishing an temporary buffer-local Org-mode environment (the
   "orgstruct-trick")

 - commenting out any new line with major-mode comment-syntax
   introduced by the Org command

I tried it in some easy cases, where all I needed to do was fixing the
global and local regexp-constants, and it worked already quite
well. But otherwise hard-coded regexp-snippets are found everywhere in
Org functions, so they just don't work out of the box in
comment-section of other major-modes. 

Therefore I'm looking for a low-level trick to introduce regexp
abstraction into Org-mode before I ask the maintainers if they want to
change the whole thing.

-- 
cheers,
Thorsten

Re: Low level trickery for changing character syntax?

[Andreas Röhler]

Am 09.04.2014 09:44, schrieb Thorsten Jolitz:
[ ... ]
>> You could define different syntax-tables and than call functions
>>
>> if type-A
>> (with-syntax-table type-A ...
>
> That looks like a promising approach, but I never worked with
> syntax-tables so I ask myself:
>
> Is it possible to redefine characters "^", "$" and "*" in a syntax-table
> in such a way that the same hardcoded regexp, e.g.
>
>    ,------------------
>    | "^[*] [*]Fat[*]$"
>    `------------------
>
> matches "* *Fat*" when called (with-syntax-table type-A ...), but
> matches e.g. "// # *Fat*//" when called (with-syntax-table type-B ...)?
>

Easiest would be a regexp matching both followed by an exclude depending from type - the mentioned negation...

Re: Low level trickery for changing character syntax?

[Stefan Monnier]

> Is it possible to redefine characters "^", "$" and "*" in a syntax-table
> in such a way that the same hardcoded regexp, e.g.

No.  Syntax-tables have no influence on those regexp elements.
They only affect things like \<, \>, \_<, \_>, \sC, \Sc and some character
ranges (like [[:word:]]).


        Stefan

Re: Org Minor Mode (was Re: Low level trickery for changing character syntax?)

[Stefan Monnier]

> The major "flaw" of Org-mode that inhibits its use as minor-mode is the
> wide-spread use of hard-coded regexps, i.e. regexps of this form
> ,--------------
> | "^\\*+ ... $"
> `--------------

Then, define some new org-header-regexp (defaults to "^\\*+") and change
all org*.el files to make use of it.


        Stefan

Re: Org Minor Mode (was Re: Low level trickery for changing character syntax?)

[Tassilo Horn]

Thorsten Jolitz <tjolitz@gmail.com> writes:

Hi Thorsten,

> The major "flaw" of Org-mode that inhibits its use as minor-mode is
> the wide-spread use of hard-coded regexps, i.e. regexps of this form
>
> ,--------------
> | "^\\*+ ... $"
> `--------------
>
> in many many variants all over the place. Those three elements "^"
> (bol), "$" (eol) and "*" (star) are not portable,

Yeah, I've quickly looked into org.el, and that's indeed very much
hard-coded.

You could add around-advices to the basic regexp functions like
`looking-at' and `re-search-forward' that modify the given regex.
Changing "^something" to "^;; something" is easy, but if you also want
to change the outlining character from * to something else, it will
become quite hard.

Changing org.el is also quite hard.  I mean, you could define

  org-prefix-rx: ""
  org-bullet-rx: "\\*"
  org-heading-start-rx: (concat org-prefix-rx "\\(" org-bullet-rx "\\)+")

somehow systematically so that in theory, lisp modes could set
org-prefix-rx to ";; " and be done.  However, it's quite unlikely that
in the future, every quick regex in some org function will be composed
of all those predefined parts in the correct way.

Maybe a better approach was to have some `org-rx' macro that's like `rx'
except that `bol' and `line-start' would mean (concat "^" org-prefix-rx)
instead of just "^", and there was some additional `*' element that
would mean org-bullet-rx.  Then the org-heading-start-rx could be
defined as (org-rx bol (group (+ *))) which is a bit less of a pain.

Bye,
Tassilo

Re: Low level trickery for changing character syntax?

[Thorsten Jolitz]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> Is it possible to redefine characters "^", "$" and "*" in a syntax-table
>> in such a way that the same hardcoded regexp, e.g.
>
> No.  Syntax-tables have no influence on those regexp elements.
> They only affect things like \<, \>, \_<, \_>, \sC, \Sc and some character
> ranges (like [[:word:]]).

Ok, thanks, then I will try to improve the things I'm working on and
then give it a try with the Org maintainers, hoping that the gains of
the required changes to Org-mode appear greater than the costs to them.

-- 
cheers,
Thorsten

Re: Org Minor Mode (was Re: Low level trickery for changing character syntax?)

[Thorsten Jolitz]

Tassilo Horn <tsdh@gnu.org> writes:

Hi Tassilo,

> Thorsten Jolitz <tjolitz@gmail.com> writes:
>
> Hi Thorsten,
>
>> The major "flaw" of Org-mode that inhibits its use as minor-mode is
>> the wide-spread use of hard-coded regexps, i.e. regexps of this form
>>
>> ,--------------
>> | "^\\*+ ... $"
>> `--------------
>>
>> in many many variants all over the place. Those three elements "^"
>> (bol), "$" (eol) and "*" (star) are not portable,
>
> Yeah, I've quickly looked into org.el, and that's indeed very much
> hard-coded.
>
> You could add around-advices to the basic regexp functions like
> `looking-at' and `re-search-forward' that modify the given regex.
> Changing "^something" to "^;; something" is easy, but if you also want
> to change the outlining character from * to something else, it will
> become quite hard.
>
> Changing org.el is also quite hard.  I mean, you could define
>
>   org-prefix-rx: ""
>   org-bullet-rx: "\\*"
>   org-heading-start-rx: (concat org-prefix-rx "\\(" org-bullet-rx "\\)+")
>
> somehow systematically so that in theory, lisp modes could set
> org-prefix-rx to ";; " and be done.  However, it's quite unlikely that
> in the future, every quick regex in some org function will be composed
> of all those predefined parts in the correct way.

outshine manages this major-mode specific calculation of outline-regexp
and outline-level quite well already, and fixing/converting regexp
constants or even regexps set to buffer-local vars is doable too, in
fact I have already functions for this in omm.el. 

The real problem are the hundreds of hardcoded regexp snippets in
countless functions, that are written in so many different styles ...

And its not only giant file org.el, its orgtbl.el, org-agenda, and many
others too. 

Achieving what I want with advising functions seems quite difficult, but
maybe its the only viable alternative. 

> Maybe a better approach was to have some `org-rx' macro that's like `rx'
> except that `bol' and `line-start' would mean (concat "^" org-prefix-rx)
> instead of just "^", and there was some additional `*' element that
> would mean org-bullet-rx.  Then the org-heading-start-rx could be
> defined as (org-rx bol (group (+ *))) which is a bit less of a pain.

Exactly what I thought, and why I implemented drx.el, which is almost
finished. 

Here is the long docstring of the main function `drx' and as an
attachment the more than 70 ERT test that describe the possibilities and
syntax better than any documentation could:

,---------------------------------------------------------------------
| (defun drx (rgxp &optional bolp stars eolp enclosing &rest rgxps)
|   "Make regexp combining RGXP and optional RGXPS.
| 
| With BOLP non-nil, add 'drx-BOL' at beginning of regexp, with EOLP
| non-nil add 'drx-EOL' at end of regexp.
| 
| STARS, when non-nil, uses 'drx-STAR' and encloses and repeats it.
| 
| ENCLOSING, when non-nil, takes RGXP and optional RGXPS and combines,
| encloses and repeats them.
| 
| While BOLP and EOLP are switches that don't do nothing when nil and
| insert whatever value 'drx-BOL' and 'drx-EOL' are set to when
| non-nil, both arguments STARS and ENCLOSING take either symbols,
| numbers, strings or (nested) lists as values and act conditional on
| the type.
| 
| All the following 'atomic' argument values are valid for both STARS
| and ENCLOSING but with a slightly different meaning:
| 
| STARS: repeat 'drx-STAR' (without enclosing) conditional on argument
| value
| 
| ENCLOSING: repeat enclosed combination of RGXP and RGXPS conditional
| on argument value
| 
|   - nil :: do nothing (no repeater, no enclosing)
| 
|   - t :: (and any other symbol w/o special meaning) repeat once
| 
|   - n :: (number) repeat n times {n}
| 
|   - \"n\" :: (number-as-string) repeat n times {n}
| 
|   - \"n,\" :: (string) repeat >= n times {n,}
| 
|   - \",m\" :: (string) repeat <= m times {,m}
| 
|   - \"n,m\" :: (string) repeat n to m times {n,m}
|        
|   - \"?\" :: (string) repeat with ?
| 
|   - \"*\" :: (string) repeat with *
| 
|   - \"+\" :: (string) repeat with +
| 
|   - \"??\" :: (string) repeat with ??
| 
|   - \"*?\" :: (string) repeat with *?
| 
|   - \"+?\" :: (string) repeat with +?
| 
|   - \"xyz\" :: (any other string) repeat once
| 
| Note that atomic values t, 1 and \"1\" should give the same
| results ('repeat once' or 'enclose and repeat once', depending on
| the context) and thus be interchangeable.
| 
| These atomic values can be wrapped in a list and change their
| meaning then. In a list of length 1 they specify 'enclose element
| first, apply repeater then'. In a list of lenght > 1 the specifier
| in the car applies to the combination of all elements, while each of
| the specifiers in the cdr applies to one element only. In the case
| of argument STAR, an element is always 'drx-STAR'. In the case of
| argument ENCLOSING, a non-nil optional argument RGXPS represents the
| list of elements, each of them being a regexp string.
| 
| Here are two calls of 'drx' with interchanged list arguments to
| STARS and ENCLOSING and their return values, demonstrating the
| above:
| 
|  ,------------------------------------------------------
|  | (drx \"foo\" t '(nil t (2)) t '(t nil (2))
|  |      \"bar\" \"loo\")
|  | \"^\\(\\*\\)\\(\\*\\)\\{2\\}\\(foobar\\(loo\\)\\{2\\}\\)$\"
|  `------------------------------------------------------
| 
|  ,------------------------------------------------------
|  | (drx \"foo\" t '(t nil (2)) t '(nil t (2))
|  |       \"bar\" \"loo\")
|  | \"^\\(\\*\\(\\*\\)\\{2\\}\\)foo\\(bar\\)\\(loo\\)\\{2\\}$\"
|  `------------------------------------------------------
| 
| Many more usage examples with their expected outcome can be found as
| ERT tests in the test-section of drx.el and should be consulted in
| doubt.
| 
| There are a few symbols with special meaning as values of the
| ENCLOSING argument (when used as atomic argument or as car of a list
| argument), namely:
|  
|   - alt :: Concat and enclose RGXP and RGXPS as regexp alternatives.
|            Eventually add drx-BOL/STARS and drx-EOL before
|            first/after last alternative.
| 
|   - grp :: Concat and enclose RGXP and RGXPS. Eventually add
|              drx-BOL, STARS and drx-EOL as first/second/last group.
| 
|   - shy :: Concat and enclose RGXP and RGXPS as shy regexp
|            groups. Eventually add drx-BOL, STARS and drx-EOL as
|            first/second/last group.
| 
|   - app :: like 'grp', but rather append RGXP and RGXPS instead
|               of enclosing them if they are already regexp groups
|               themselves.
| 
| They create regexp groups but don't apply repeaters to them."
`---------------------------------------------------------------------


,-------------------------------------------------------------------------------
| ;;;; ERT Tests
| 
| ;; return identity
| (ert-deftest drx-test-1 ()
|   "Test return values of function `drx'.
| Assumes the following variable definitions:
| 
|  (defvar drx-BOL \"^\"
|    \"Special character that signals BOL in regexps.\")
| 
|  (defvar drx-EOL \"$\"
|    \"Special character that signals EOL in regexps.\")
| 
|  (defvar drx-STAR (regexp-quote \"*\")
|    \"Special character that signals headline(-level) in regexps.\")"
|   (should (equal
|            (drx "foo")
|            "foo")))
| 
| ;; add drx-BOL
| (ert-deftest drx-test-2 ()
|   "See docstring of `drx-test-1'."
|   (should (equal
|            (drx "foo" t)
|            "^foo")))
| 
| ;; append drx-EOL
| (ert-deftest drx-test-3 ()
|   "See docstring of `drx-test-1'."
|   (should (equal
|            (drx "foo" nil nil t)
|            "foo$")))
| 
| ;; add drx-BOL and append drx-EOL
| (ert-deftest drx-test-4 ()
|   "See docstring of `drx-test-1'."
|   (should (equal
|            (drx "foo" t nil t)
|            "^foo$")))
| 
| ;; add drx-BOL and append drx-EOL
| ;; and add drx-STAR with default quantifier
| (ert-deftest drx-test-5 ()
|   "See docstring of `drx-test-1'."
|   (should (equal
|            (drx "foo" t t t)
|            "^\\*foo$")))
| 
| (ert-deftest drx-test-6 ()
|   "See docstring of `drx-test-1'."
|   (should (equal
|            (drx "foo" t 'bar t)
|            "^\\*foo$")))
| 
| (ert-deftest drx-test-7 ()
|   "See docstring of `drx-test-1'."
|   (should (equal
|            (drx "foo" t "bar" t)
|            "^\\*foo$")))
| 
| ;; add drx-BOL and append drx-EOL
| ;; and add drx-STAR with specified quantifiers
| (ert-deftest drx-test-8 ()
|   "See docstring of `drx-test-1'."
|   (should (equal
|            (drx "foo" t "0," t)
|            "^\\*\\{0,\\}foo$")))
| 
| (ert-deftest drx-test-9 ()
|   "See docstring of `drx-test-1'."
|   (should (equal
|            (drx "foo" t ",8" t)
|            "^\\*\\{,8\\}foo$")))
| 
| (ert-deftest drx-test-10 ()
|   "See docstring of `drx-test-1'."
|   (should (equal
|            (drx "foo" t "1,8" t)
|            "^\\*\\{1,8\\}foo$")))
| 
| (ert-deftest drx-test-11 ()
|   "See docstring of `drx-test-1'."
|   (should (equal
|            (drx "foo" t "*" t)
|            "^\\**foo$")))
| 
| (ert-deftest drx-test-12 ()
|   "See docstring of `drx-test-1'."
|   (should (equal
|            (drx "foo" t "+" t)
|            "^\\*+foo$")))
| 
| (ert-deftest drx-test-13 ()
|   "See docstring of `drx-test-1'."
|   (should (equal
|            (drx "foo" t "?" t)
|            "^\\*?foo$")))
| 
| (ert-deftest drx-test-14 ()
|   "See docstring of `drx-test-1'."
|   (should (equal
|            (drx "foo" t "*?" t)
|            "^\\**?foo$")))
| 
| (ert-deftest drx-test-15 ()
|   "See docstring of `drx-test-1'."
|   (should (equal
|            (drx "foo" t "+?" t)
|            "^\\*+?foo$")))
| 
| ;; add drx-STAR with specified quantifier list
| (ert-deftest drx-test-16 ()
|   "See docstring of `drx-test-1'."
|   (should (equal
|            (drx "foo" t "??" t)
|            "^\\*??foo$")))
| 
| (ert-deftest drx-test-17 ()
|   "See docstring of `drx-test-1'."
|   (should (equal
|            (drx "foo" nil '(nil) nil)
|            "foo")))
| 
| (ert-deftest drx-test-18 ()
|   "See docstring of `drx-test-1'."
|   (should (equal
|            (drx "foo" nil '(t) nil)
|            "\\(\\)foo")))
| 
| (ert-deftest drx-test-19 ()
|   "See docstring of `drx-test-1'."
|   (should (equal
|            (drx "foo" nil '(t nil) nil)
|            "\\(\\*\\)foo")))
| 
| (ert-deftest drx-test-20 ()
|   "See docstring of `drx-test-1'."
|   (should (equal
|            (drx "foo" nil '(t 1) nil)
|            "\\(\\*\\)foo")))
| 
| (ert-deftest drx-test-21 ()
|   "See docstring of `drx-test-1'."
|   (should (equal
|            (drx "foo" nil '(t t) nil)
|            "\\(\\(\\*\\)\\)foo")))
| 
| (ert-deftest drx-test-22 ()
|   "See docstring of `drx-test-1'."
|   (should (equal
|            (drx "foo" nil '(nil "+") nil)
|            "\\*+foo")))
| 
| (ert-deftest drx-test-23 ()
|   "See docstring of `drx-test-1'."
|   (should (equal
|            (drx "foo" nil '(nil "*") nil)
|            "\\**foo")))
| 
| (ert-deftest drx-test-24 ()
|   "See docstring of `drx-test-1'."
|   (should (equal
|            (drx "foo" nil '(nil "?") nil)
|            "\\*?foo")))
| 
| (ert-deftest drx-test-25 ()
|   "See docstring of `drx-test-1'."
|   (should (equal
|            (drx "foo" nil '(nil "+?") nil)
|            "\\*+?foo")))
| 
| (ert-deftest drx-test-26 ()
|   "See docstring of `drx-test-1'."
|   (should (equal
|            (drx "foo" nil '(nil "*?") nil)
|            "\\**?foo")))
| 
| (ert-deftest drx-test-27 ()
|   "See docstring of `drx-test-1'."
|   (should (equal
|            (drx "foo" nil '(nil "??") nil)
|            "\\*??foo")))
| 
| (ert-deftest drx-test-28 ()
|   "See docstring of `drx-test-1'."
|   (should (equal
|            (drx "foo" nil '(nil "1,") nil)
|            "\\*\\{1,\\}foo")))
| 
| (ert-deftest drx-test-29 ()
|   "See docstring of `drx-test-1'."
|   (should (equal
|            (drx "foo" nil '(nil ("1,")) nil)
|            "\\(\\*\\)\\{1,\\}foo")))
| 
| (ert-deftest drx-test-30 ()
|   "See docstring of `drx-test-1'."
|   (should (equal
|            (drx "foo" nil '(nil ",2") nil)
|            "\\*\\{,2\\}foo")))
| 
| (ert-deftest drx-test-31 ()
|   "See docstring of `drx-test-1'."
|   (should (equal
|            (drx "foo" nil '(nil (",2")) nil)
|            "\\(\\*\\)\\{,2\\}foo")))
| 
| (ert-deftest drx-test-32 ()
|   "See docstring of `drx-test-1'."
|   (should (equal
|            (drx "foo" nil '(nil "1,2") nil)
|            "\\*\\{1,2\\}foo")))
| 
| (ert-deftest drx-test-33 ()
|   "See docstring of `drx-test-1'."
|   (should (equal
|            (drx "foo" nil '(nil ("1,2")) nil)
|            "\\(\\*\\)\\{1,2\\}foo")))
| 
| (ert-deftest drx-test-34 ()
|   "See docstring of `drx-test-1'."
|   (should (equal
|            (drx "foo" nil '(nil (2)) nil)
|            "\\(\\*\\)\\{2\\}foo")))
| 
| (ert-deftest drx-test-35 ()
|   "See docstring of `drx-test-1'."
|   (should (equal
|            (drx "foo" nil '(nil 2) nil)
|            "\\*\\{2\\}foo")))
| 
| (ert-deftest drx-test-36 ()
|   "See docstring of `drx-test-1'."
|   (should (equal
|            (drx "foo" nil '(nil (1)) nil)
|            "\\(\\*\\)foo")))
| 
| (ert-deftest drx-test-37 ()
|   "See docstring of `drx-test-1'."
|   (should (equal
|            (drx "foo" nil '(nil 1) nil)
|            "\\*foo")))
| 
| (ert-deftest drx-test-38 ()
|   "See docstring of `drx-test-1'."
|   (should (equal
|            (drx "foo" nil '(nil 2 ("2,3") "3") nil)
|            "\\*\\{2\\}\\(\\*\\)\\{2,3\\}\\*\\{3\\}foo")))
| 
| (ert-deftest drx-test-39 ()
|   "See docstring of `drx-test-1'."
|   (should (equal
|            (drx "foo" nil '(",3" (2) ("2,3") 3) nil)
|            "\\(\\(\\*\\)\\{2\\}\\(\\*\\)\\{2,3\\}\\*\\{3\\}\\\)\\{,3\\}foo")))
| 
| ;; temporarily change BOL and EOL e.g. using CSS comment syntax
| (ert-deftest drx-test-40 ()
|   "See docstring of `drx-test-1'."
|   (should (equal
|            (let ((drx-BOL (concat "^" (regexp-quote "/* ")))
|                  (drx-EOL (concat (regexp-quote "*/") "$")))
|              (drx "foo" t nil t))
|            "^/\\* foo\\*/$")))
| 
| ;; temporarily change STAR using Elisp syntax
| (ert-deftest drx-test-41 ()
|   "See docstring of `drx-test-1'."
|   (should (equal
|            (let ((drx-STAR ";"))
|              (drx " foo" nil 2))
|            ";\\{2\\} foo")))
| 
| (ert-deftest drx-test-42 ()
|   "See docstring of `drx-test-1'."
|   (should (equal
|            (let ((drx-BOL "^;;")
|                  (drx-STAR ";"))
|              (drx " foo" t '(2 2) nil))
|            "^;;\\(;\\{2\\}\\)\\{2\\} foo")))
| 
| ;; temporarily change STAR to whitespace syntax
| (ert-deftest drx-test-43 ()
|   "See docstring of `drx-test-1'."
|   (should (equal
|            (let ((drx-STAR "[ \t]"))
|              (drx " foo" t "*"))
|            "^[  ]* foo")))
| 
| ;; enclose rgxp
| (ert-deftest drx-test-44 ()
|   "See docstring of `drx-test-1'."
|   (should (equal
|            (drx "foo" nil nil nil t)
|            "\\(foo\\)")))
| 
| (ert-deftest drx-test-45 ()
|   "See docstring of `drx-test-1'."
|   (should (equal
|            (drx "foo" t t t t)
|            "^\\*\\(foo\\)$")))
| 
| (ert-deftest drx-test-46 ()
|   "See docstring of `drx-test-1'."
|   (should (equal
|            (drx "foo" nil nil nil 'alt "bar")
|            "\\(foo\\|bar\\)")))
| 
| (ert-deftest drx-test-47 ()
|   "See docstring of `drx-test-1'."
|   (should (equal
|            (drx "foo" nil nil nil 'grp "bar")
|            "\\(foo\\)\\(bar\\)")))
| 
| (ert-deftest drx-test-48 ()
|   "See docstring of `drx-test-1'."
|   (should (equal
|            (drx "foo" nil nil nil 'shy "bar")
|            "\\(?:foo\\)\\(?:bar\\)")))
| 
| (ert-deftest drx-test-49 ()
|   "See docstring of `drx-test-1'."
|   (should (equal
|            (drx "foo" nil nil nil 'app "bar")
|            "\\(foo\\)\\(bar\\)")))
| 
| (ert-deftest drx-test-50 ()
|   "See docstring of `drx-test-1'."
|   (should (equal
|            (drx "foo" nil nil nil 'app "\\(bar\\)")
|            "\\(foo\\)\\(bar\\)")))
| 
| (ert-deftest drx-test-51 ()
|   "See docstring of `drx-test-1'."
|   (should (equal
|            (drx "foo" nil nil nil 'grp "\\(bar\\)")
|            "\\(foo\\)\\(\\(bar\\)\\)")))
| 
| (ert-deftest drx-test-52 ()
|   "See docstring of `drx-test-1'."
|   (should (equal
|            (drx "foo" nil nil nil 'alt "bar")
|            "\\(foo\\|bar\\)")))
| 
| (ert-deftest drx-test-53 ()
|   "See docstring of `drx-test-1'."
|   (should (equal
|            (drx "foo" nil nil nil t)
|            "\\(foo\\)")))
| 
| (ert-deftest drx-test-54 ()
|   "See docstring of `drx-test-1'."
|   (should (equal
|            (drx "foo" nil nil nil 2)
|            "\\(foo\\)\\{2\\}")))
| 
| (ert-deftest drx-test-55 ()
|   "See docstring of `drx-test-1'."
|   (should (equal
|            (drx "foo" nil nil nil "2")
|            "\\(foo\\)\\{2\\}")))
| 
| (ert-deftest drx-test-56 ()
|   "See docstring of `drx-test-1'."
|   (should (equal
|            (drx "foo" nil nil nil 1)
|            "\\(foo\\)")))
| 
| (ert-deftest drx-test-57 ()
|   "See docstring of `drx-test-1'."
|   (should (equal
|            (drx "foo" nil nil nil "1")
|            "\\(foo\\)")))
| 
| (ert-deftest drx-test-58 ()
|   "See docstring of `drx-test-1'."
|   (should (equal
|            (drx "foo" nil nil nil "1,")
|            "\\(foo\\)\\{1,\\}")))
| 
| (ert-deftest drx-test-59 ()
|   "See docstring of `drx-test-1'."
|   (should (equal
|            (drx "foo" nil nil nil ",1")
|            "\\(foo\\)\\{,1\\}")))
| 
| (ert-deftest drx-test-60 ()
|   "See docstring of `drx-test-1'."
|   (should (equal
|            (drx "foo" nil nil nil "1,3")
|            "\\(foo\\)\\{1,3\\}")))
| 
| (ert-deftest drx-test-61 ()
|   "See docstring of `drx-test-1'."
|   (should (equal
|            (drx "foo" nil nil nil "bar")
|            "\\(foo\\)")))
| 
| (ert-deftest drx-test-62 ()
|   "See docstring of `drx-test-1'."
|   (should (equal
|            (drx "foo" nil nil nil nil "bar" "loo")
|            "foobarloo")))
| 
| (ert-deftest drx-test-63 ()
|   "See docstring of `drx-test-1'."
|   (should (equal
|            (drx "foo" nil nil nil t "bar" "loo")
|            "\\(foobarloo\\)")))
| 
| (ert-deftest drx-test-64 ()
|   "See docstring of `drx-test-1'."
|   (should (equal
|            (drx "foo" nil nil nil 'grp "bar" "loo")
|            "\\(foo\\)\\(bar\\)\\(loo\\)")))
| 
| (ert-deftest drx-test-65 ()
|   "See docstring of `drx-test-1'."
|   (should (equal
|            (drx "foo" nil nil nil 'alt "bar" "loo")
|            "\\(foo\\|bar\\|loo\\)")))
| 
| (ert-deftest drx-test-66 ()
|   "See docstring of `drx-test-1'."
|   (should (equal
|            (drx "foo" t nil t 'shy "bar" "loo")
|            "^\\(?:foo\\)\\(?:bar\\)\\(?:loo\\)$")))
| 
| (ert-deftest drx-test-67 ()
|   "See docstring of `drx-test-1'."
|   (should (equal
|            (drx "foo" t t t 'app "\\(bar\\)" "loo")
|            "^\\*\\(foo\\)\\(bar\\)\\(loo\\)$")))
| 
| (ert-deftest drx-test-68 ()
|   "See docstring of `drx-test-1'."
|   (should (equal
|            (drx "foo" t 1 t 'grp "\\(bar\\)" "loo")
|            "^\\*\\(foo\\)\\(\\(bar\\)\\)\\(loo\\)$")))
| 
| (ert-deftest drx-test-69 ()
|   "See docstring of `drx-test-1'."
|   (should (equal
|            (drx "foo" t 2 t 'app "\\(bar\\)" "loo")
|            "^\\*\\{2\\}\\(foo\\)\\(bar\\)\\(loo\\)$")))
| 
| (ert-deftest drx-test-70 ()
|   "See docstring of `drx-test-1'."
|   (should (equal
|            (drx "foo" nil nil nil '(nil) "bar" "loo")
|            "foobarloo")))
| 
| (ert-deftest drx-test-71 ()
|   "See docstring of `drx-test-1'."
|   (should (equal
|            (drx "foo" nil nil nil '(nil nil) "bar" "loo")
|            "foobarloo")))
| 
| (ert-deftest drx-test-72 ()
|   "See docstring of `drx-test-1'."
|   (should (equal
|            (drx "foo" nil nil nil '(t) "bar" "loo")
|            "\\(foobarloo\\)")))
| 
| (ert-deftest drx-test-73 ()
|   "See docstring of `drx-test-1'."
|   (should (equal
|            (drx "foo" nil nil nil '(t t) "bar" "loo")
|            "\\(foo\\(bar\\)loo\\)")))
| 
| (ert-deftest drx-test-74 ()
|   "See docstring of `drx-test-1'."
|   (should (equal
|            (drx "foo" nil nil nil '(t nil t) "bar" "loo")
|            "\\(foobar\\(loo\\)\\)")))
| 
| (ert-deftest drx-test-75 ()
|   "See docstring of `drx-test-1'."
|   (should (equal
|            (drx "foo" nil nil nil '(t t t) "bar" "loo")
|            "\\(foo\\(bar\\)\\(loo\\)\\)")))
| ;;
| 
| (ert-deftest drx-test-76 ()
|   "See docstring of `drx-test-1'."
|   (should (equal
|            (drx "foo" nil nil nil '(1 1 1) "bar" "loo")
|            "\\(foo\\(bar\\)\\(loo\\)\\)")))
| 
| (ert-deftest drx-test-77 ()
|   "See docstring of `drx-test-1'."
|   (should (equal
|            (drx "foo" t '(t t t) t '(t t t) "bar" "loo")
|            "\\(foo\\(bar\\)\\(loo\\)\\)")))
| 
| (ert-deftest drx-test-78 ()
|   "See docstring of `drx-test-1'."
|   (should (equal
|            (drx "foo" 1 '(1 1 1) 1 '(1 1 1) "bar" "loo")
|            "\\(foo\\(bar\\)\\(loo\\)\\)")))
| 
| (ert-deftest drx-test-79 ()
|   "See docstring of `drx-test-1'."
|   (should (equal
|            (drx "foo" "1" '("1" "1" "1") "1" '("1" "1" "1")
|                 "bar" "loo")
|            "\\(foo\\(bar\\)\\(loo\\)\\)")))
`-------------------------------------------------------------------------------


I'm not yet done, but pretty close:

Selector: t
Passed: 75
Failed: 4 (4 unexpected)
Total:  79/79

Started at:   2014-04-09 15:41:04+0200
Finished.
Finished at:  2014-04-09 15:41:05+0200

.........................................................................FFFF..


-- 
cheers,
Thorsten