Re: regexp does not work as documented

[Bruno Haible <bruno@clisp.org>]

Chong Yidong wrote:
> BTW, here is a simplified recipe, for those who didn't download the
> attached file:
> 
> 1. Copy the following text, between the "---...----" lines, into a
>    buffer
> 
> ------------------
> # Messages français pour GNU gettext.
> # Copyright © 2006 Free Software Foundation, Inc.
> # François Pinard <pinard@iro.umontreal.ca>, 1996.
> #
> #
> msgid ""
> msgstr ""
> "Project-Id-Version: GNU gettext-tools 0.16.2-pre5\n"
> "Report-Msgid-Bugs-To: bug-gnu-gettext@gnu.org\n"
> "POT-Creation-Date: 2007-11-02 03:23+0100\n"
> "PO-Revision-Date: 2007-10-27 13:35+0200\n"
> "Last-Translator: Christophe Combelles <ccomb@free.fr>\n"
> "Language-Team: French <traduc@traduc.org>\n"
> "MIME-Version: 1.0\n"
> "Content-Type: text/plain; charset=UTF-8\n"
> "Content-Transfer-Encoding: 8bit\n"
> "Plural-Forms: nplurals=2; plural=(n > 1);\n"
> ------------------
> 
> 2. M-: (highlight-regexp "^m.*\n\\(\".*\n\\)+") RET
> 
> Note that the last two lines remain unhighlighted.

Yes. I reproduce with this simpler recipe as well. Thank you.

> I believe this bug arises because highlight-regexp uses font-lock to
> highlight the regular expression, and the font-lock engine is
> intentionally limiting the region to search for the multi-line regular
> expression.

You are right that there is a limit, but it is set to 200000:
highlight-regexp is aliased to hi-lock-face-buffer, which asks for the
arguments and calls hi-lock-set-pattern. hi-lock-set-pattern does little
more than applying a margin of 100000 and calling re-search-forward.

I believe the origin of the bug is deeper, because
  - the limit of 100000 is way larger than the little snippet you posted,
  - I originally observed the bug in po-mode (part of GNU gettext), in
    a function po-find-span-of-entry which essentially only calls
    re-search-backward and re-search-forward.

> OTOH, I don't see what we can do about this problem.  Maybe we could add
> a note to the docstring of highlight-regexp saying that multi-line
> regular expressions are problematic?

Can someone help me find a workaround, then? If not, I would have to give up
maintaining po-mode as part of GNU gettext. Said function is central in
Emacs po-mode (everything else relies on it), and if multi-line regular
expressions don't work, I don't know how this function could be rewritten.

Bruno