filladapt mode stops filling after lengthy amount of emacs uptime

filladapt mode stops filling after lengthy amount of emacs uptime

[John Owens]

Greetings, I've had this filladapt mode problem for several versions
of emacsen, and it hasn't got any better.

Basically when I have an emacs running for a long time (days), 
eventually filladapt just stops working. Restarting emacs always
fixes the problem. The error is below; it beeps and flags "regular
expression too big" when I go past 70-something characters in a line
in mail mode (for instance). I am running the most recent Carbon Emacs
version of emacs, 22.0.92.1, on OS X (PPC) 10.4.9.

Anything I can do to help debugging?

JDO

Debugger entered--Lisp error: (invalid-regexp "Regular expression too big")
  looking-at( ... )
  fill-nobreak-p()
  fill-move-to-break-point(279)
  #[nil "[big long complex string]‡" [fill-prefix give-up justify fc
auto-fill-inhibit-regexp adaptive-fill-mode nil current-justification
current-fill-column left beginning-of-line looking-at (full center right)
unjustify-current-line "" fill-context-prefix backward-paragraph 1
forward-paragraph string-match "\\`[ 	]*\\'" regexp-quote 0 move-to-column
fill-move-to-break-point " " re-search-forward t " 	" justify-current-line
prefix fill-indent-according-to-mode after-prefix fill-point comment-start-skip
limit prev-column comment-line-break-function] 5] nil)
  filladapt-funcall(do-auto-fill)
  byte-code("\bƒ[bytecode]" [filladapt-mode fill-prefix retval
filladapt-inside-filladapt comment-multi-line adaptive-fill-regexp nil t
filladapt-adapt filladapt-funcall do-auto-fill throw done adaptive-fill-mode
paragraph-ignore-fill-prefix] 7)
  do-auto-fill()
  mail-mode-auto-fill()

Re: filladapt mode stops filling after lengthy amount of emacs uptime

[Chong Yidong]

John Owens <john_owens@yahoo.com> writes:

> Basically when I have an emacs running for a long time (days), 
> eventually filladapt just stops working. Restarting emacs always
> fixes the problem. The error is below; it beeps and flags "regular
> expression too big" when I go past 70-something characters in a line
> in mail mode (for instance). I am running the most recent Carbon Emacs
> version of emacs, 22.0.92.1, on OS X (PPC) 10.4.9.
>
> Debugger entered--Lisp error: (invalid-regexp "Regular expression too big")
>   looking-at( ... )
>   fill-nobreak-p()

What is the string argument to "looking-at" that you snipped?

Re: filladapt mode stops filling after lengthy amount of emacs uptime

[John Owens]

Chong Yidong <cyd <at> stupidchicken.com> writes:

> John Owens <john_owens <at> yahoo.com> writes:
> 
> > Basically when I have an emacs running for a long time (days), 
> > eventually filladapt just stops working. Restarting emacs always
> > fixes the problem. The error is below; it beeps and flags "regular
> > expression too big" when I go past 70-something characters in a line
> > in mail mode (for instance). I am running the most recent Carbon Emacs
> > version of emacs, 22.0.92.1, on OS X (PPC) 10.4.9.
> >
> > Debugger entered--Lisp error: (invalid-regexp "Regular expression too big")
> >   looking-at( ... )
> >   fill-nobreak-p()
> 
> What is the string argument to "looking-at" that you snipped?
> 

OK. It really is a long regexp! Emacs has been up for days now and
the error popped back up again. Backtrace at:

http://tinyurl.com/2dpnra

Suggestions?

(rms is likely right in saying it's a filladapt problem.)

JDO

Re: filladapt mode stops filling after lengthy amount of emacs uptime

[Stefan Monnier]

> OK. It really is a long regexp! Emacs has been up for days now and
> the error popped back up again. Backtrace at:

> http://tinyurl.com/2dpnra

> Suggestions?

The regexp is mostly composed of a repetition of

  "\\(--[[][[]\\([^]]*\\)]\\([[]\\([^]]*\\)]\\|\\)]\\|"

so there's most likely a place in the filladapt code (or some related code)
which adds this every once in a while without making sure that it hasn't
added it already.


        Stefan

Re: filladapt mode stops filling after lengthy amount of emacs uptime

[Chong Yidong]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> The regexp is mostly composed of a repetition of
>
>   "\\(--[[][[]\\([^]]*\\)]\\([[]\\([^]]*\\)]\\|\\)]\\|"
>
> so there's most likely a place in the filladapt code (or some related code)
> which adds this every once in a while without making sure that it hasn't
> added it already.

Judging from the backtrace, this is happening to either the variable
sentence-end or paragraph-start.  However, I can't see any place in
the Emacs sources that set either of these variables in such a
dangerous way, nor do I see such a regexp in use anywhere.

John, could you check if your customizations, site-start files, and
third-party elisp packages set sentence-end or paragraph-start
anywhere, or if they contain this regexp?

Re: filladapt mode stops filling after lengthy amount of emacs uptime

[John Owens]

Chong Yidong <cyd <at> stupidchicken.com> writes:
> 
> John, could you check if your customizations, site-start files, and
> third-party elisp packages set sentence-end or paragraph-start
> anywhere, or if they contain this regexp?

Thanks for the suggestion. I am definitely an elisp rookie, so
forgive any mistakes here.

In .emacs, I have

(setq sentence-end-double-space nil)

Don't think that's an issue. (That's used in filladapt.)

In erc.el (an IRC client, I could totally get rid of this), I have

(defun erc-mode ()
  ...
  (make-variable-buffer-local 'paragraph-start)
  ...
  (setq paragraph-start (concat "\\(" (regexp-quote (erc-prompt)) "\\)"))

but since I'm never in erc-mode, that probably isn't an issue either.

More possibly the culprit is the MIME editing facility in SEMI, the MIME
handling library (used in my mailer Wanderlust). In it:

(defun turn-on-mime-edit ()
    ...
    (setq paragraph-start
	  (regexp-or mime-edit-single-part-tag-regexp
		     paragraph-start))

Here's that regexp. 

(defconst mime-edit-single-part-tag-regexp
  "--[[][[]\\([^]]*\\)]\\([[]\\([^]]*\\)]\\|\\)]"
  "*Regexp of MIME tag in the form of [[CONTENT-TYPE][ENCODING]].")

Does that seem like it might be the problem? (SEMI's problem?)

Really, many thanks for all of your help in tracking this down.

JDO

Re: filladapt mode stops filling after lengthy amount of emacs uptime

[Davis Herring]

> More possibly the culprit is the MIME editing facility in SEMI, the MIME
> handling library (used in my mailer Wanderlust). In it:
>
> (defun turn-on-mime-edit ()
>     ...
>     (setq paragraph-start
> 	  (regexp-or mime-edit-single-part-tag-regexp
> 		     paragraph-start))
>
> Here's that regexp.
>
> (defconst mime-edit-single-part-tag-regexp
>   "--[[][[]\\([^]]*\\)]\\([[]\\([^]]*\\)]\\|\\)]"
>   "*Regexp of MIME tag in the form of [[CONTENT-TYPE][ENCODING]].")
>
> Does that seem like it might be the problem? (SEMI's problem?)

That sounds very much like the problem, like `turn-on-mime-edit' is
getting called repeatedly in the same buffer.  Try adding

(message "turn-on-mime-edit(buffer=%s,pslen=%s)"
         (current-buffer) (length paragraph-start))

to that function (evaluating the `defun' after that library has been
loaded, of course).  Then you can test this hypothesis.  If it is being
called repeatedly, either there is a bug elsewhere that calls it when it's
not necessary, or that function should be modified to only modify
paragraph-start if it has not done so already (perhaps by setting a flag
or consulting the existing value).

Davis

-- 
This product is sold by volume, not by mass.  If it appears too dense or
too sparse, it is because mass-energy conversion has occurred during
shipping.

Re: filladapt mode stops filling after lengthy amount of emacs uptime

[Chong Yidong]

John Owens <john_owens@yahoo.com> writes:

> More possibly the culprit is the MIME editing facility in SEMI, the MIME
> handling library (used in my mailer Wanderlust). In it:
>
> (defun turn-on-mime-edit ()
>     ...
>     (setq paragraph-start
> 	  (regexp-or mime-edit-single-part-tag-regexp
> 		     paragraph-start))
>
> Here's that regexp. 
>
> (defconst mime-edit-single-part-tag-regexp
>   "--[[][[]\\([^]]*\\)]\\([[]\\([^]]*\\)]\\|\\)]"
>   "*Regexp of MIME tag in the form of [[CONTENT-TYPE][ENCODING]].")
>
> Does that seem like it might be the problem? (SEMI's problem?)

That's definitely it.  Please contact the Wanderlust developers and
tell them about this problem.

Re: filladapt mode stops filling after lengthy amount of emacs uptime

[Stefan Monnier]

> In erc.el (an IRC client, I could totally get rid of this), I have

> (defun erc-mode ()
>   ...
>   (make-variable-buffer-local 'paragraph-start)
>   ...
>   (setq paragraph-start (concat "\\(" (regexp-quote (erc-prompt)) "\\)"))

Please tell the ERC developer that they're confusing make-local-variable
with make-variable-buffer-local.

> (defun turn-on-mime-edit ()
>     ...
>     (setq paragraph-start
> 	  (regexp-or mime-edit-single-part-tag-regexp
> 		     paragraph-start))

> Here's that regexp. 

> (defconst mime-edit-single-part-tag-regexp
>   "--[[][[]\\([^]]*\\)]\\([[]\\([^]]*\\)]\\|\\)]"
>   "*Regexp of MIME tag in the form of [[CONTENT-TYPE][ENCODING]].")

Obviously, that's it.  Please tell them about the problem and about the fact
that they should call make-local-variable on paragraph-start.


        Stefan

Re: filladapt mode stops filling after lengthy amount of emacs uptime

[Andreas Schwab]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> In erc.el (an IRC client, I could totally get rid of this), I have
>
>> (defun erc-mode ()
>>   ...
>>   (make-variable-buffer-local 'paragraph-start)
>>   ...
>>   (setq paragraph-start (concat "\\(" (regexp-quote (erc-prompt)) "\\)"))
>
> Please tell the ERC developer that they're confusing make-local-variable
> with make-variable-buffer-local.

I cannot find theses lines in lisp/erc/erc.el.

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
PGP key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: filladapt mode stops filling after lengthy amount of emacs uptime

[Stefan Monnier]

>>> In erc.el (an IRC client, I could totally get rid of this), I have
>> 
>>> (defun erc-mode ()
>>> ...
>>> (make-variable-buffer-local 'paragraph-start)
>>> ...
>>> (setq paragraph-start (concat "\\(" (regexp-quote (erc-prompt)) "\\)"))
>> 
>> Please tell the ERC developer that they're confusing make-local-variable
>> with make-variable-buffer-local.

> I cannot find theses lines in lisp/erc/erc.el.

Good.  He was probably looking at an older version, or something like that.


        Stefan

Re: filladapt mode stops filling after lengthy amount of emacs uptime

[Michael Olson]

[-- Attachment #1.1: Type: text/plain, Size: 990 bytes --]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>>>> In erc.el (an IRC client, I could totally get rid of this), I have
>>> 
>>>> (defun erc-mode ()
>>>> ...
>>>> (make-variable-buffer-local 'paragraph-start)
>>>> ...
>>>> (setq paragraph-start (concat "\\(" (regexp-quote (erc-prompt)) "\\)"))
>>> 
>>> Please tell the ERC developer that they're confusing make-local-variable
>>> with make-variable-buffer-local.
>
>> I cannot find theses lines in lisp/erc/erc.el.
>
> Good.  He was probably looking at an older version, or something like that.

Yep, this is fixed in the version of ERC which is bundled with Emacs.

-- 
Michael Olson -- FSF Associate Member #652 -- http://www.mwolson.org/
Interests: Lisp, text markup, protocols -- Jabber: mwolson_at_hcoop.net
  /` |\ | | | Projects: Emacs, Muse, ERC, EMMS, Planner, ErBot, DVC
 |_] | \| |_| Reclaim your digital rights by eliminating DRM.
      See http://www.defectivebydesign.org/what_is_drm for details.

[-- Attachment #1.2: Type: application/pgp-signature, Size: 188 bytes --]

[-- Attachment #2: Type: text/plain, Size: 142 bytes --]

_______________________________________________
Emacs-devel mailing list
Emacs-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/emacs-devel

Re: filladapt mode stops filling after lengthy amount of emacs uptime

[Davis Herring]

> OK. It really is a long regexp! Emacs has been up for days now and
> the error popped back up again. Backtrace at:
>
> http://tinyurl.com/2dpnra
>
> Suggestions?

I've got one for Emacs: it should quit its whining.  307 nested groups is
not complicated by today's standards!

Davis

-- 
This product is sold by volume, not by mass.  If it appears too dense or
too sparse, it is because mass-energy conversion has occurred during
shipping.

Re: filladapt mode stops filling after lengthy amount of emacs uptime

[Richard Stallman]

    I've got one for Emacs: it should quit its whining.

That is rather insulting, which is not a way to be helpful.

Re: filladapt mode stops filling after lengthy amount of emacs uptime

[Davis Herring]

>     I've got one for Emacs: it should quit its whining.
>
> That is rather insulting, which is not a way to be helpful.

It was a joke!  The actual import of that message was the groups in the
regexp nested to 307 levels; Stefan noted the repetitious nature of the
rest of it.  Combined with the more recent message that a function
`regexp-or' that could generate that kind of output was likely involved,
it helped track down the problem.

Emacs itself has no feelings, of course, so I imagined that I would not be
misinterpreted.  Apologies to anyone actually offended.

Davis

-- 
This product is sold by volume, not by mass.  If it appears too dense or
too sparse, it is because mass-energy conversion has occurred during
shipping.

Re: filladapt mode stops filling after lengthy amount of emacs uptime

[David Kastrup]

"Davis Herring" <herring@lanl.gov> writes:

>>     I've got one for Emacs: it should quit its whining.
>>
>> That is rather insulting, which is not a way to be helpful.
>
> It was a joke!  The actual import of that message was the groups in
> the regexp nested to 307 levels; Stefan noted the repetitious nature
> of the rest of it.  Combined with the more recent message that a
> function `regexp-or' that could generate that kind of output was
> likely involved, it helped track down the problem.
>
> Emacs itself has no feelings, of course, so I imagined that I would
> not be misinterpreted.  Apologies to anyone actually offended.

Emacs has no feelings?  You've never seen it bluster its way out of a
lost game of gomoku, right?

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: filladapt mode stops filling after lengthy amount of emacs uptime

[Davis Herring]

>> Emacs itself has no feelings, of course, so I imagined that I would
>> not be misinterpreted.  Apologies to anyone actually offended.
>
> Emacs has no feelings?  You've never seen it bluster its way out of a
> lost game of gomoku, right?

My local implementation of gomoku does not lose; it calls abort() if the
user (presumably with a debugger or so) interferes with the game in some
way to prevent it from winning.

Davis

-- 
This product is sold by volume, not by mass.  If it appears too dense or
too sparse, it is because mass-energy conversion has occurred during
shipping.

Re: filladapt mode stops filling after lengthy amount of emacs uptime

[Richard Stallman]

Sorry I misunderstood your joke.

Re: filladapt mode stops filling after lengthy amount of emacs uptime

[John Owens]

Richard Stallman <rms <at> gnu.org> writes:

> 
>     I've got one for Emacs: it should quit its whining.
> 
> That is rather insulting, which is not a way to be helpful.

As the author of the original post, I certainly didn't take 
offense; I actually thought Davis's comment was pretty funny.

Again I thank the nice folks who have helped us track this
down. I've sent a note to the SEMI developers and hopefully
we can fix it up.

JDO

Re: filladapt mode stops filling after lengthy amount of emacs uptime

[Richard Stallman]

    Anything I can do to help debugging?

    JDO

    Debugger entered--Lisp error: (invalid-regexp "Regular expression too big")
      looking-at( ... )

The debugging has to start with debugging filladapt, but filladapt is
not part of Emacs.  Debugging this may or may not lead you to an Emacs
bug.

I suggest starting by looking at that regexp.  Is it really too big?
If so, you could try to see why filladapt made it.

Re: filladapt mode stops filling after lengthy amount of emacs uptime

[John Owens]

Howdy, I've got zero response from the SEMI developers.
So I hope maybe I can enlist the help of smart folks here
to help me fix it myself. The bad function is below:

The answer is to make paragraph-start a local variable
(make-local-variable). How is this done? Single call to
make-local-variable in this function (and that will work)?
Where should I put it?

A one-sentence explanation of what this does would be very
useful for me (I can see having a local variable would be
good, but how long does a local variable last, "local 
variable" to me means "local for the lifetime of the function" -
doesn't this function complete immediately after you call it?

Thanks -

JDO

=========================

(defun turn-on-mime-edit ()
  "Unconditionally turn on MIME-Edit mode."
  (interactive)
  (if mime-edit-mode-flag
      (error "You are already editing a MIME message.")
    (setq mime-edit-mode-flag t)

    ;; Set transfer level into mode line
    ;;
    (setq mime-transfer-level-string
 	  (mime-encoding-name mime-transfer-level 'not-omit))
    (force-mode-line-update)

    ;; Define menu for XEmacs.
    (if (featurep 'xemacs)
	(mime-edit-define-menu-for-xemacs)
      )

    (enable-invisible)

    ;; I don't care about saving these.
    (setq paragraph-start
	  (regexp-or mime-edit-single-part-tag-regexp
		     paragraph-start))
    (setq paragraph-separate
	  (regexp-or mime-edit-single-part-tag-regexp
		     paragraph-separate))
    (run-hooks 'mime-edit-mode-hook)
    (message
     "%s"
     (substitute-command-keys
      "Type \\[mime-edit-exit] to exit MIME mode, and type \\[mime-edit-help] to
get help."))
    ))

Re: filladapt mode stops filling after lengthy amount of emacs uptime

[Stefan Monnier]

> A one-sentence explanation of what this does would be very
> useful for me (I can see having a local variable would be
> good, but how long does a local variable last, "local 
> variable" to me means "local for the lifetime of the function" -
> doesn't this function complete immediately after you call it?

C-h f make-local-variable RET explains that it doesn't make a function-local
variable (these are created with `let'), but a buffer-local one.
The buffer-local variable so-created will last until the buffer dies or
until kill-all-local-variables is called in this buffer (this is called by
major-mode functions as the first thing to do when setting up a major mode).


        Stefan