bug#7181: 24.0.50; args-out-of-range while reverting C++ header file

bug#7181: 24.0.50; args-out-of-range while reverting C++ header file

[Oscar Fuentes]

This bug report will be sent to the Free Software Foundation,
not to your local site managers!
Please write in English if possible, because the Emacs maintainers
usually do not have translators to read other languages for them.

Your bug report will be posted to the bug-gnu-emacs@gnu.org mailing list,
and to the gnu.emacs.bug news group.

Please describe exactly what actions triggered the bug
and the precise symptoms of the bug.  If you can, give
a recipe starting from `emacs -Q':

emacs -Q
C-x C-f some-moderately-big-header-file.h
<make some big modification on disk to the file>
M-x revert-buffer
Lisp error: (args-out-of-range 580 36722)

The buffer is left semi-reverted, on an edited state. Doing a M-x
revert-buffer again works ok.

Setting debug-on-error to t shows this:

Debugger entered--Lisp error: (args-out-of-range 580 36722)
  remove-text-properties(580 36722 (category))
  c-clear-char-property-with-value-function(1 36722 category c-cpp-delimiter)
  c-neutralize-syntax-in-and-mark-CPP(174 174 36141)
  c-after-change(174 174 36141)
  insert-file-contents("/home/oscar/dev/idb/lp0/ffi.h" t nil nil t)
  revert-buffer(t)
  call-interactively(revert-buffer t nil)
  execute-extended-command(nil)
  call-interactively(execute-extended-command nil nil)

If Emacs crashed, and you have the Emacs process in the gdb debugger,
please include the output from the following gdb commands:
    `bt full' and `xbacktrace'.
For information about debugging Emacs, please read the file
/usr/local/share/emacs/24.0.50/etc/DEBUG.


In GNU Emacs 24.0.50.1 (x86_64-unknown-linux-gnu, X toolkit)
 of 2010-09-16 on qcore
Windowing system distributor `The X.Org Foundation', version 11.0.60900000
configured using `configure  '--without-toolkit-scroll-bars' '--with-x-toolkit=lucid''

Important settings:
  value of $LC_ALL: nil
  value of $LC_COLLATE: nil
  value of $LC_CTYPE: nil
  value of $LC_MESSAGES: nil
  value of $LC_MONETARY: nil
  value of $LC_NUMERIC: nil
  value of $LC_TIME: nil
  value of $LANG: nil
  value of $XMODIFIERS: nil
  locale-coding-system: nil
  default enable-multibyte-characters: t

Major mode: Debugger

Minor modes in effect:
  tooltip-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Recent input:
C-x C-f f f i . h <return> M-x r e v e r t SPC b u 
f f e r <return> y e s <return> M-x <up> <return> y 
e s <return> SPC C-_ M-x s e t SPC v a r i <tab> <return> 
d e b u <tab> o n <tab> e r r <tab> <return> t <return> 
M-x r e v e r t SPC b u f f e r <return> y e s <return> 
M-x r e v e r t SPC b u f f e r <return> y e s <return> 
<up> <up> <up> <up> <up> <up> <up> <up> <up> <up> C-SPC 
<down> <down> <down> <down> <down> <down> <down> <down> 
<down> <down> M-w <help-echo> <help-echo> <help-echo> 
<help-echo> <help-echo> <menu-bar> <help-menu> <se
nd-emacs-bug-report>

Recent messages:
For information about GNU Emacs and the GNU system, type C-h C-a.
Loading vc-git...done
c-clear-char-property-with-value-function: Args out of range: 580, 36722
Undo!
Entering debugger...
Mark set

Load-path shadows:
None found.

Features:
(shadow sort gnus-util mail-extr message sendmail rfc822 mml mml-sec
mm-decode mm-bodies mm-encode mail-parse rfc2231 rfc2047 rfc2045
ietf-drums mm-util mail-prsvr mailabbrev mail-utils gmm-utils mailheader
emacsbug help-mode view debug cus-edit wid-edit cus-start cus-load
help-fns vc-git cc-mode cc-fonts easymenu cc-menus cc-cmds cc-styles
cc-align cc-engine cc-vars cc-defs regexp-opt tooltip ediff-hook
vc-hooks lisp-float-type mwheel x-win x-dnd tool-bar dnd fontset image
fringe lisp-mode register page menu-bar rfn-eshadow timer select
scroll-bar mldrag mouse jit-lock font-lock syntax facemenu font-core
frame cham georgian utf-8-lang misc-lang vietnamese tibetan thai
tai-viet lao korean japanese hebrew greek romanian slovak czech european
ethiopic indian cyrillic chinese case-table epa-hook jka-cmpr-hook help
simple abbrev loaddefs button minibuffer faces cus-face files
text-properties overlay md5 base64 format env code-pages mule custom
widget hashtable-print-readable backquote make-network-process
dynamic-setting font-render-setting x-toolkit x multi-tty emacs)

bug#7181: 24.0.50; args-out-of-range while reverting C++ header file

[Eli Zaretskii]

> From: ofv@wanadoo.es (Oscar Fuentes)
> Date: Sat, 09 Oct 2010 06:30:32 +0200
> Cc: 
> 
> emacs -Q
> C-x C-f some-moderately-big-header-file.h
> <make some big modification on disk to the file>
> M-x revert-buffer
> Lisp error: (args-out-of-range 580 36722)
> 
> The buffer is left semi-reverted, on an edited state. Doing a M-x
> revert-buffer again works ok.
> 
> Setting debug-on-error to t shows this:
> 
> Debugger entered--Lisp error: (args-out-of-range 580 36722)
>   remove-text-properties(580 36722 (category))

Can you verify that the problem happens because one or both argument
positions of remove-text-properties don't exist due to the changes?

bug#7181: 24.0.50; args-out-of-range while reverting C++ header file

[Óscar Fuentes]

Eli Zaretskii <eliz@gnu.org> writes:

>> emacs -Q
>> C-x C-f some-moderately-big-header-file.h
>> <make some big modification on disk to the file>
>> M-x revert-buffer
>> Lisp error: (args-out-of-range 580 36722)
>> 
>> The buffer is left semi-reverted, on an edited state. Doing a M-x
>> revert-buffer again works ok.
>> 
>> Setting debug-on-error to t shows this:
>> 
>> Debugger entered--Lisp error: (args-out-of-range 580 36722)
>>   remove-text-properties(580 36722 (category))
>
> Can you verify that the problem happens because one or both argument
> positions of remove-text-properties don't exist due to the changes?

Essentially, the end position which is out of range corresponds to the
size of the *previous* contents of the file, not of the current
file. For instance:

While visiting file ffi.h with size 26675, replace it on disk with
another version of 36721 bytes, revert-buffer, (args-out-of-range 580
26676).

If we are visiting the long version of ffi.h (the one with 36721 bytes)
and replace it on disk with the short version, then (args-out-of-range
580 36722).

On both cases the buffer is left with 580 bytes of content, part of them
belonging to the beginning of the file and the other part to the end of
the file. The first byte missing from the buffer after revert-buffer
fails is the first character which differs from one version to another
of the file.

I'll try to reduce a test case file.

bug#7181: 24.0.50; args-out-of-range while reverting C++ header file

[Óscar Fuentes]

[-- Attachment #1: Type: text/plain, Size: 457 bytes --]

With the attached files the problem can be reproduced predictability:

Save the attached files b.h.master and b.h.next to a directory and cd to
it.

$ emacs -Q &
$ cp b.h.master b.h
C-x C-f b.h
$ cp b.h.next b.h
M-x revert-buffer

and the args-out-of-range error pops:

c-clear-char-property-with-value-function: Args out of range: 166, 453

I was unable to try with the latest Emacs sources because the build is
broken. My Emacs is from the 16 September.


[-- Attachment #2: b.h.master --]
[-- Type: application/octet-stream, Size: 452 bytes --]

#ifndef BASICOPS_H_F902161C
#define BASICOPS_H_F902161C

#include "ffi.h"
#include <time.h>

LP0_NEW_TYPE_DEC_EXPORTED(unsigned char);
LP0_NEW_TYPE_DEC_EXPORTED(tm);

namespace lp0 {

struct Int : public OpBase0<int> {};

struct CastCharToStringOP : public OpBase1<std::string, char> {
  static void w(std::string *r, char v) {
    std::string t;
    t += v;
    FFIRetClass(r, t);
  }
};

} // namespace lp0

#endif /* #ifndef BASICOPS_H_F902161C */ 

[-- Attachment #3: b.h.next --]
[-- Type: application/octet-stream, Size: 349 bytes --]

#ifndef BASICOPS_H_F902161C
#define BASICOPS_H_F902161C

#include "ffi.h"
#include <time.h>

namespace lp0 {

template <typename Derived, typename Base>
struct DynamicCastOp : public OpBase1<Derived*, Base*>
{
  static Derived* w(Base *p) {
    return dynamic_cast<Derived*>(p);
  }
};

} // namespace lp0

#endif /* #ifndef BASICOPS_H_F902161C */ 

Re: bug#7181: 24.0.50; args-out-of-range while reverting C++ header file

[Alan Mackenzie]

Hi, Óscar,

On Sat, Oct 09, 2010 at 03:56:39PM +0200, Óscar Fuentes wrote:
> With the attached files the problem can be reproduced predictability:
> 
> Save the attached files b.h.master and b.h.next to a directory and cd to
> it.
> 
> $ emacs -Q &
> $ cp b.h.master b.h
> C-x C-f b.h
> $ cp b.h.next b.h
> M-x revert-buffer
> 
> and the args-out-of-range error pops:
> 
> c-clear-char-property-with-value-function: Args out of range: 166, 453
> 
> I was unable to try with the latest Emacs sources because the build is
> broken. My Emacs is from the 16 September.
> 

Thanks for the recipe.  I turned on debug-on-error and got this
traceback:

Debugger entered--Lisp error: (args-out-of-range 166 453)
  remove-text-properties(166 453 (category))
  c-clear-char-property-with-value-function(1 453 category c-cpp-delimiter)
  c-neutralize-syntax-in-and-mark-CPP(94 94 286)
  c-after-change(94 94 286) <<======================================
  insert-file-contents("/home/acm/b.h" t nil nil t)
  revert-buffer(t)
  call-interactively(revert-buffer nil nil)

c-after-change is on the after-change-functions hook.  The values of BEG
END OLD-LEN it is being passed look utterly wrong.  BEG and END should
delimit the entire buffer and OLD-LEN should be the length of the
previous buffer.  Surely?

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#7181: after-change-functions hook arguments (was: bug#7181: 24.0.50; args-out-of-range while reverting C++ header file)

[Óscar Fuentes]

Alan Mackenzie <acm@muc.de> writes:

> Thanks for the recipe.  I turned on debug-on-error and got this
> traceback:
>
> Debugger entered--Lisp error: (args-out-of-range 166 453)
>   remove-text-properties(166 453 (category))
>   c-clear-char-property-with-value-function(1 453 category c-cpp-delimiter)
>   c-neutralize-syntax-in-and-mark-CPP(94 94 286)
>   c-after-change(94 94 286) <<======================================
>   insert-file-contents("/home/acm/b.h" t nil nil t)
>   revert-buffer(t)
>   call-interactively(revert-buffer nil nil)
>
> c-after-change is on the after-change-functions hook.  The values of BEG
> END OLD-LEN it is being passed look utterly wrong.  BEG and END should
> delimit the entire buffer and OLD-LEN should be the length of the
> previous buffer.  Surely?

Seems that insert-file-contents is trying to be smart and determine
which parts of the contents are shared among the old and the new
text. On this case b.h.next and b.h.master differ only on the central
region of text. The arguments passed to c-after-change indicates a
deletion from position 94 to position 94+286=380.

I don't know if that is correct, as the transformation is not a deletion
but a replacement of the middle text segment of the buffer with a text
chunk of different lenght, and the doc is not clear about that case.

after-change-functions hook arguments (was: bug#7181: 24.0.50; args-out-of-range while reverting C++ header file)

[Óscar Fuentes]

Alan Mackenzie <acm@muc.de> writes:

> Thanks for the recipe.  I turned on debug-on-error and got this
> traceback:
>
> Debugger entered--Lisp error: (args-out-of-range 166 453)
>   remove-text-properties(166 453 (category))
>   c-clear-char-property-with-value-function(1 453 category c-cpp-delimiter)
>   c-neutralize-syntax-in-and-mark-CPP(94 94 286)
>   c-after-change(94 94 286) <<======================================
>   insert-file-contents("/home/acm/b.h" t nil nil t)
>   revert-buffer(t)
>   call-interactively(revert-buffer nil nil)
>
> c-after-change is on the after-change-functions hook.  The values of BEG
> END OLD-LEN it is being passed look utterly wrong.  BEG and END should
> delimit the entire buffer and OLD-LEN should be the length of the
> previous buffer.  Surely?

Seems that insert-file-contents is trying to be smart and determine
which parts of the contents are shared among the old and the new
text. On this case b.h.next and b.h.master differ only on the central
region of text. The arguments passed to c-after-change indicates a
deletion from position 94 to position 94+286=380.

I don't know if that is correct, as the transformation is not a deletion
but a replacement of the middle text segment of the buffer with a text
chunk of different lenght, and the doc is not clear about that case.

bug#7181: 24.0.50; args-out-of-range while reverting C++ header file

[Stefan Monnier]

>> With the attached files the problem can be reproduced predictability:
>> 
>> Save the attached files b.h.master and b.h.next to a directory and cd to
>> it.
>> 
>> $ emacs -Q &
>> $ cp b.h.master b.h
>> C-x C-f b.h
>> $ cp b.h.next b.h
>> M-x revert-buffer
>> 
>> and the args-out-of-range error pops:
>> 
>> c-clear-char-property-with-value-function: Args out of range: 166, 453
>> 
>> I was unable to try with the latest Emacs sources because the build is
>> broken. My Emacs is from the 16 September.
>> 

> Thanks for the recipe.  I turned on debug-on-error and got this
> traceback:

> Debugger entered--Lisp error: (args-out-of-range 166 453)
>   remove-text-properties(166 453 (category))
>   c-clear-char-property-with-value-function(1 453 category c-cpp-delimiter)
>   c-neutralize-syntax-in-and-mark-CPP(94 94 286)
>   c-after-change(94 94 286) <<======================================
>   insert-file-contents("/home/acm/b.h" t nil nil t)
>   revert-buffer(t)
>   call-interactively(revert-buffer nil nil)

> c-after-change is on the after-change-functions hook.  The values of BEG
> END OLD-LEN it is being passed look utterly wrong.  BEG and END should
> delimit the entire buffer and OLD-LEN should be the length of the
> previous buffer.  Surely?

No, here the after-change is called right after removing the chars that
are different, and just before inserting the new (different) chars.
You can reproduce the same error without reverting:

emacs -Q b.h.master
M-x c-mode
C-u 93 C-f
C-SPC
C-u 286 C-f
C-w


        Stefan

Re: bug#7181: 24.0.50; args-out-of-range while reverting C++ header file

[Stefan Monnier]

>> With the attached files the problem can be reproduced predictability:
>> 
>> Save the attached files b.h.master and b.h.next to a directory and cd to
>> it.
>> 
>> $ emacs -Q &
>> $ cp b.h.master b.h
>> C-x C-f b.h
>> $ cp b.h.next b.h
>> M-x revert-buffer
>> 
>> and the args-out-of-range error pops:
>> 
>> c-clear-char-property-with-value-function: Args out of range: 166, 453
>> 
>> I was unable to try with the latest Emacs sources because the build is
>> broken. My Emacs is from the 16 September.
>> 

> Thanks for the recipe.  I turned on debug-on-error and got this
> traceback:

> Debugger entered--Lisp error: (args-out-of-range 166 453)
>   remove-text-properties(166 453 (category))
>   c-clear-char-property-with-value-function(1 453 category c-cpp-delimiter)
>   c-neutralize-syntax-in-and-mark-CPP(94 94 286)
>   c-after-change(94 94 286) <<======================================
>   insert-file-contents("/home/acm/b.h" t nil nil t)
>   revert-buffer(t)
>   call-interactively(revert-buffer nil nil)

> c-after-change is on the after-change-functions hook.  The values of BEG
> END OLD-LEN it is being passed look utterly wrong.  BEG and END should
> delimit the entire buffer and OLD-LEN should be the length of the
> previous buffer.  Surely?

No, here the after-change is called right after removing the chars that
are different, and just before inserting the new (different) chars.
You can reproduce the same error without reverting:

emacs -Q b.h.master
M-x c-mode
C-u 93 C-f
C-SPC
C-u 286 C-f
C-w


        Stefan

bug#7181: 24.0.50; args-out-of-range while reverting C++ header file

[Alan Mackenzie]

Hi, Óscar,

On Sat, Oct 09, 2010 at 03:56:39PM +0200, Óscar Fuentes wrote:
> With the attached files the problem can be reproduced predictability:
> 
> Save the attached files b.h.master and b.h.next to a directory and cd to
> it.
> 
> $ emacs -Q &
> $ cp b.h.master b.h
> C-x C-f b.h
> $ cp b.h.next b.h
> M-x revert-buffer
> 
> and the args-out-of-range error pops:
> 
> c-clear-char-property-with-value-function: Args out of range: 166, 453
> 
> I was unable to try with the latest Emacs sources because the build is
> broken. My Emacs is from the 16 September.
> 

Thanks for the recipe.  I turned on debug-on-error and got this
traceback:

Debugger entered--Lisp error: (args-out-of-range 166 453)
  remove-text-properties(166 453 (category))
  c-clear-char-property-with-value-function(1 453 category c-cpp-delimiter)
  c-neutralize-syntax-in-and-mark-CPP(94 94 286)
  c-after-change(94 94 286) <<======================================
  insert-file-contents("/home/acm/b.h" t nil nil t)
  revert-buffer(t)
  call-interactively(revert-buffer nil nil)

c-after-change is on the after-change-functions hook.  The values of BEG
END OLD-LEN it is being passed look utterly wrong.  BEG and END should
delimit the entire buffer and OLD-LEN should be the length of the
previous buffer.  Surely?

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#7181: 24.0.50; args-out-of-range while reverting C++ header file. PATCH.

[Alan Mackenzie]

Hi, Óscar,

On Sat, Oct 09, 2010 at 03:56:39PM +0200, Óscar Fuentes wrote:
> With the attached files the problem can be reproduced predictability:
> 
> Save the attached files b.h.master and b.h.next to a directory and cd to
> it.
> 
> $ emacs -Q &
> $ cp b.h.master b.h
> C-x C-f b.h
> $ cp b.h.next b.h
> M-x revert-buffer
> 
> and the args-out-of-range error pops:
> 
> c-clear-char-property-with-value-function: Args out of range: 166, 453
> 
> I was unable to try with the latest Emacs sources because the build is
> broken. My Emacs is from the 16 September.
> 

Please try the following patch and confirm that it works.  I'll get it
committed just as soon as I manage to connect to the new server:




*** cc-mode.el.~3~	2010-10-08 20:27:13.000000000 +0000
--- cc-mode.el	2010-10-10 13:46:28.074200752 +0000
***************
*** 1029,1038 ****
  			    (buffer-substring-no-properties type-pos term-pos)
  			    (buffer-substring-no-properties beg end)))))))
  
- 	;; (c-new-BEG c-new-END) will be the region to fontify.  It may become
- 	;; larger than (beg end).
- 	(setq c-new-BEG beg
- 	      c-new-END end)
  	(if c-get-state-before-change-functions
  	    (mapc (lambda (fn)
  		    (funcall fn beg end))
--- 1029,1034 ----
***************
*** 1086,1091 ****
--- 1082,1091 ----
  	(when c-recognize-<>-arglists
  	  (c-after-change-check-<>-operators beg end))
  
+ 	;; (c-new-BEG c-new-END) will be the region to fontify.  It may become
+ 	;; larger than (beg end).
+ 	(setq c-new-BEG beg
+ 	      c-new-END end)
  	(if c-before-font-lock-function
  	    (save-excursion
  	      (funcall c-before-font-lock-function beg end old-len)))))))

      

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#7181: 24.0.50; args-out-of-range while reverting C++ header file. PATCH.

[Óscar Fuentes]

Alan Mackenzie <acm@muc.de> writes:

> Please try the following patch and confirm that it works.  I'll get it
> committed just as soon as I manage to connect to the new server:

[snip]

Yep, the error is gone. Thank you, Alan.

Re: 24.0.50; args-out-of-range while reverting C++ header file

[Alan Mackenzie]

Hi, Oscar

On Sat, Oct 09, 2010 at 06:30:32AM +0200, Oscar Fuentes wrote:

> emacs -Q
> C-x C-f some-moderately-big-header-file.h
> <make some big modification on disk to the file>
> M-x revert-buffer
> Lisp error: (args-out-of-range 580 36722)

I do this, but can't reproduce the error.  Can you please be precise
about which file.h triggers the problem (send me a copy of it if it's
outside of Emacs, please) and what the "big modification" is.

I built the latest Emacs-24 last night.  I tried it with loading
.../emacs/src/lisp.h, then

M-5 M->
M-<
C-w
M-x revert-buffer<CR>yes

So, what file (?ffi.h) was it, please, and what did you do to it?

This error message has been thrown before in the medium past, but I
think this is something new.

> The buffer is left semi-reverted, on an edited state. Doing a M-x
> revert-buffer again works ok.

> Setting debug-on-error to t shows this:

> Debugger entered--Lisp error: (args-out-of-range 580 36722)
>   remove-text-properties(580 36722 (category))
>   c-clear-char-property-with-value-function(1 36722 category c-cpp-delimiter)
>   c-neutralize-syntax-in-and-mark-CPP(174 174 36141)
>   c-after-change(174 174 36141)
>   insert-file-contents("/home/oscar/dev/idb/lp0/ffi.h" t nil nil t)
>   revert-buffer(t)
>   call-interactively(revert-buffer t nil)
>   execute-extended-command(nil)
>   call-interactively(execute-extended-command nil nil)

> In GNU Emacs 24.0.50.1 (x86_64-unknown-linux-gnu, X toolkit)
>  of 2010-09-16 on qcore

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: 24.0.50; args-out-of-range while reverting C++ header file

[Miles Bader]

Alan Mackenzie <acm@muc.de> writes:
>> C-x C-f some-moderately-big-header-file.h
>> <make some big modification on disk to the file>
>> M-x revert-buffer
>> Lisp error: (args-out-of-range 580 36722)
>
> I do this, but can't reproduce the error.  Can you please be precise
> about which file.h triggers the problem (send me a copy of it if it's
> outside of Emacs, please) and what the "big modification" is.
>
> I built the latest Emacs-24 last night.  I tried it with loading
> .../emacs/src/lisp.h, then

Hmm, I'm getting similar behavior, but I'm not using "revert-buffer".

I think the basic scenario is:

  1) I have a buffer visiting config.h (autoconf-generated)

  2) I re-run make or something, which causes config.h to be regenerated
  and changed (the latter's important, as autoconf doesn't touch it if
  the contents don't actually change)

  3) I visit config.h again with find-file; since it's already in a
  buffer, but has changed, Emacs asks the usual question in this
  situation:

     File config.h changed on disk.  Reread from disk? (yes or no) 

  4) I answer "yes RET", and then I get an error:

     c-clear-char-property-with-value-function: Args out of range: 464, 3922

  [and don't end up visiting the vile]

  5) I use find-file again on config.h; this time, Emacs asks:

     File config.h changed on disk.  Discard your edits? (yes or no)

  [despite the fact that I never actually made any changes to that
  buffer; I guess the failed update changed it.]

  6) I answer "yes RET", and this time am left in the config.h buffer,
  properly updated from disk.

I think this recipe can be fairly easily followed (you can just touch
the file or something, of course, instead of getting autoconf to change
it).

Thanks,

-Miles

-- 
gravity a demanding master ... soft soft snow

Re: 24.0.50; args-out-of-range while reverting C++ header file

[Oscar Fuentes]

Miles Bader <miles@gnu.org> writes:

[snip]

> I think this recipe can be fairly easily followed (you can just touch
> the file or something, of course, instead of getting autoconf to change
> it).

Not really. The file on disk must be different from the file on the
buffer, and the file must have certain features (as it doesn't happen
with all files.) I just posted a test case to bug#7181 with attached
files for reproducing the problem.