bug#5570: Emacs 23.1.91.1 C++ mode "ESC C-q" and TAB indenting issue.

bug#5570: Emacs 23.1.91.1 C++ mode "ESC C-q" and TAB indenting issue.

[Bob]

[-- Attachment #1: Type: text/plain, Size: 785 bytes --]

As I work with text in C++ mode the indentation begins to deteriorate. For
example, TAB on the class line wrongly indents it:

 

    template<class _Enum>

      class Build<TL::End,_Enum>

 

I work-around the problem by writing the buffer, killing the buffer, then
visiting the file and re-indenting with TAB on the class line to get the
expected indentation:

 

    template<class _Enum>

    class Build<TL::End,_Enum>

 

Same thing happens with ESC C-q.

 

My impression is there is internal state which doesn't get cleared as I work
the text toward a syntax correct unit, causing incorrect indentation.  The
bad state is ultimately cleared by re-visiting the file. 

 

Sorry I can't come up with something reproducible.  

 

Has anyone else seen this issue?

 

 

      


[-- Attachment #2: Type: text/html, Size: 3259 bytes --]

bug#5570: Emacs 23.1.91.1 C++ mode "ESC C-q" and TAB indenting issue.

[Chong Yidong]

Hi Alan,

Looks like this is related to Bug#5490.  WDYT?


"Bob" <purchasebyemail1@comcast.net> writes:

> As I work with text in C++ mode the indentation begins to deteriorate. For
> example, TAB on the class line wrongly indents it:
>
>     template<class _Enum>
>       class Build<TL::End,_Enum>
>
> I work-around the problem by writing the buffer, killing the buffer, then
> visiting the file and re-indenting with TAB on the class line to get the
> expected indentation:
>
>     template<class _Enum>
>     class Build<TL::End,_Enum>
>
> Same thing happens with ESC C-q.
>
> My impression is there is internal state which doesn't get cleared as I work
> the text toward a syntax correct unit, causing incorrect indentation.  The bad
> state is ultimately cleared by re-visiting the file.
>
> Sorry I can't come up with something reproducible.
>
> Has anyone else seen this issue?

bug#5570: Emacs 23.1.91.1 C++ mode "ESC C-q" and TAB indenting issue.

[Chong Yidong]

Forwarding an email with pertinent info to the bug tracker, and to Alan
the CC-mode maintainer:


"Bob" <purchasebyemail1@comcast.net> writes:

> Yes, it does look related to 5490. I can respond to:
>
>> (i) Have you ever noticed this bug happening on earlier Emacsen, in
>>  particular Emacs 23.1?
>
> Yes, it happened in 23.1 also.
>
>> (ii) Does this bug happen every time you use Emacs this long, or only
>>  sometimes?
>
> For me it happens within minutes of use. My C++ files are HEAVILY templated
> and the issue appears to be related to '<' and '>' of template args, e.g.
>
> typedef IF<
>  MemberOf<a,b>::value,
>  IF<
>    Length<c,d>::value,
>    true,
>     false
>  >::Result,
>  false
>   >::Result  Result;
>
> I will have several of these, one after the other, and be editing
> them. By accident I'll leave out a ',' or a '>' or kill & yank lines
> with similar error. After minutes of such editing, the indentation
> gets wacked. Sometimes by TAB indentation on a prior line the problem
> corrects. Sometimes I have to write the buffer and re-visit it.
>
>>(iv) Are all C(++) buffers in this state, or only some?  Any correlation
>>  with how many edits have been done on a buffer, or how long it's been
>>  loaded?
>
> Just the buffer with the '<' and '>' in it.

bug#5570: Emacs 23.1.91.1 C++ mode "ESC C-q" and TAB indenting issue.

[Alan Mackenzie]

Hi, Bob and Yidong,

On Sun, Feb 14, 2010 at 12:09:39AM -0500, Chong Yidong wrote:
> Forwarding an email with pertinent info to the bug tracker, and to Alan
> the CC-mode maintainer:


> "Bob" <purchasebyemail1@comcast.net> writes:

> > Yes, it does look related to 5490. I can respond to:

I have the horrible suspicion that this is different from 5490.  :-(

> >> (i) Have you ever noticed this bug happening on earlier Emacsen, in
> >> particular Emacs 23.1?

> > Yes, it happened in 23.1 also.

> >> (ii) Does this bug happen every time you use Emacs this long, or
> >> only sometimes?

> > For me it happens within minutes of use. My C++ files are HEAVILY
> > templated and the issue appears to be related to '<' and '>' of
> > template args, e.g.

> > typedef IF<
> >  MemberOf<a,b>::value,
> >  IF<
> >    Length<c,d>::value,
> >    true,
> >     false
> >  >::Result,
> >  false
> >   >::Result  Result;

OK, template brackets are difficult, because they're so difficult to
distinguish from less/greater than.

Bob, seeing as how the bug hits you "withing minutes", is there any
chance you could try and get it reproducible?  Using a dribble file might
be useful (see the page "Checklist" in the Emacs manual; you'd use this
with something like '(open-dribble-file "~/dribble")' at the beginning of
your Emacs session).

If you manage to get it reproducible, could you send me the C++ file
(which I will regard as confidential if needed) together with the
sequence of commands (e.g. the dribble file) which triggers the bug.
Also appreciated would be a dump of your CC Mode configuration, generated
by C-c C-b.

> > I will have several of these, one after the other, and be editing
> > them. By accident I'll leave out a ',' or a '>' or kill & yank lines
> > with similar error. After minutes of such editing, the indentation
> > gets wacked. Sometimes by TAB indentation on a prior line the problem
> > corrects. Sometimes I have to write the buffer and re-visit it.

One other thing.  Would you please try M-: (c-state-cache-init) when in
this state, and tell me if it resolves the badness.  Thanks!

> >>(iv) Are all C(++) buffers in this state, or only some?  Any correlation
> >>  with how many edits have been done on a buffer, or how long it's been
> >>  loaded?

> > Just the buffer with the '<' and '>' in it.

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#5570: Emacs 23.1.91.1 C++ mode "ESC C-q" and TAB indenting issue.

[Bob]

More stuff:

With the emacs I've had going since last week I have several buffers where
the
indentation is incorrect. I validate by writing the buffer, then in a fresh
emacs,
visit the file, go to the lines with incorrect indentation, apply TAB and
the
result is the correct indentation. Ok, nothing new there. 

This is new: After applying TAB in the 2nd emacs to correct the indentation
I write the file, then go back to the original buffer and re-visit the file
with "C-x C-f" "ESC-n".
Emacs confirms the file has changed, I reply "yes" and the visible buffer
becomes truncated
at the line where I TAB'ed to reformat:

The saved file is:

    template<
      class _TypeKind,
      class _EStore = typename ScalarVectorSelector<_TypeKind>::Result
      bool  _fixed  = TL::MemberOf<
	DataTypeTrait<GetDataType<_TypeKind>::Result>::TAbout,
	EnumDTP::FixedPacked
	>::value
      >

The re-visited buffer now shows:

    template<
      class _TypeKind,
      class _EStore = typename ScalarVectorSelector<_TypeKind>::Result
      bool  _fixed  = TL::MemberOf<
	   
EOF  <--------------- NO MORE TEXT!

The original incorrect indentation was something like this (it got lost when
I revisited):

    template<
      class _TypeKind,
      class _EStore = typename ScalarVectorSelector<_TypeKind>::Result
      bool  _fixed  = TL::MemberOf<
	        DataTypeTrait<GetDataType<_TypeKind>::Result>::TAbout,
	                      EnumDTP::FixedPacked
	                      >::value
              >

Now, I kill the truncated buffer, re-visit (the original emacs) and I see
correct indentation of the saved file, so I can continue work :) in the
original emacs.


-----Original Message-----
From: Alan Mackenzie [mailto:acm@muc.de] 
Sent: Sunday, February 14, 2010 12:10 PM
To: Chong Yidong
Cc: Bob; 5570@debbugs.gnu.org
Subject: Re: bug#5570: Emacs 23.1.91.1 C++ mode "ESC C-q" and TAB indenting
issue.

Hi, Bob and Yidong,

On Sun, Feb 14, 2010 at 12:09:39AM -0500, Chong Yidong wrote:
> Forwarding an email with pertinent info to the bug tracker, and to Alan
> the CC-mode maintainer:


> "Bob" <purchasebyemail1@comcast.net> writes:

> > Yes, it does look related to 5490. I can respond to:

I have the horrible suspicion that this is different from 5490.  :-(

> >> (i) Have you ever noticed this bug happening on earlier Emacsen, in
> >> particular Emacs 23.1?

> > Yes, it happened in 23.1 also.

> >> (ii) Does this bug happen every time you use Emacs this long, or
> >> only sometimes?

> > For me it happens within minutes of use. My C++ files are HEAVILY
> > templated and the issue appears to be related to '<' and '>' of
> > template args, e.g.

> > typedef IF<
> >  MemberOf<a,b>::value,
> >  IF<
> >    Length<c,d>::value,
> >    true,
> >     false
> >  >::Result,
> >  false
> >   >::Result  Result;

OK, template brackets are difficult, because they're so difficult to
distinguish from less/greater than.

Bob, seeing as how the bug hits you "withing minutes", is there any
chance you could try and get it reproducible?  Using a dribble file might
be useful (see the page "Checklist" in the Emacs manual; you'd use this
with something like '(open-dribble-file "~/dribble")' at the beginning of
your Emacs session).

If you manage to get it reproducible, could you send me the C++ file
(which I will regard as confidential if needed) together with the
sequence of commands (e.g. the dribble file) which triggers the bug.
Also appreciated would be a dump of your CC Mode configuration, generated
by C-c C-b.

> > I will have several of these, one after the other, and be editing
> > them. By accident I'll leave out a ',' or a '>' or kill & yank lines
> > with similar error. After minutes of such editing, the indentation
> > gets wacked. Sometimes by TAB indentation on a prior line the problem
> > corrects. Sometimes I have to write the buffer and re-visit it.

One other thing.  Would you please try M-: (c-state-cache-init) when in
this state, and tell me if it resolves the badness.  Thanks!

> >>(iv) Are all C(++) buffers in this state, or only some?  Any correlation
> >>  with how many edits have been done on a buffer, or how long it's been
> >>  loaded?

> > Just the buffer with the '<' and '>' in it.

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#5570: Emacs 23.1.91.1 C++ mode "ESC C-q" and TAB indenting issue.

[Alan Mackenzie]

Hi, Bob,

On Mon, Feb 15, 2010 at 09:14:52AM -0800, Bob wrote:
> Alan,

> I am now running two emacs 23.1.91.1, one I started last week with many
> files open in which I "work" etc., and one I just started with dribbling
> "on" to edit a file I have seen the issue in. After half-hours editing in
> this one, nothing bad has happened. I temporarily jumped back to the "work"
> session to add a few needed lines and wammy! the problem appeared after
> adding 7 lines:

Could you be a little bit more exact in exactly what you did here.  I
have just fired up emacs 23.1.91, typed in 

    namespace foo
    {

(in column 0), then typed in the the lines that you've identified as
causing the error:

>   template<class _DataType>
>   struct SelectInternalConvertableToVectorType
>   {
>     typedef typename DataTypeTrait<_DataType>::TAbout	TAbout;
>
>     typedef TL::IF<TL::MemberOf<TAbout,EnumDTP::Internal>::value>
> TInternal;
>     typedef TL::IF<
> 	TL::MemberOf<TAbout,EnumDTP::HasVectorType>::value
> 	>					   TConvertableToVectorType;
> 	
>     typedef TL::IF<
>       (TInternal::value &&
> TConvertableToVectorType::value),_DataType,TL::False
>       >::Result						Result;
> };   <------------------------ Appears in column 1

However, that "};" for me appeared in the correct column, not in column
0.

> I wrote the buffer, started a 3rd emacs, went to the line, hit TAB and the
> indentation
> corrected:

That, at least, is refreshing.

>   };   <------------------------ Correct.

> Back in the original session TAB still places the '{' in column 1.

It doesn't for me.  But I _have_ seen this error happening.

So, could you please try and give me a precise recipe for how to
reproduce the error, down at the level of what the buffer should look
like before inserting these lines, and which keys to press to insert
them.  Was I correct in guessing that you typed in the lines when you
said you "added" them, as opposed to, say, C-y?

> I am now attempting to try your c-state-cache-init and find this is not
> defined.

That's exactly

    M-: (c-state-cache-init)

, including the parens round the word.  Are you certain you typed it
correctly?  It should be there in Emacs 23.1.91.

> Please advise what to do. I've attached "describe-bindings" if that helps. 

> Thanks,
> Bob

> PS "C-c C-b" runs "c-submit-bug-report" - is that what you expect?

Yes.  It produces a dump of CC Mode's config as part of a mail buffer.
Cut and paste it into your own mail client.

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#5570: Emacs 23.1.91.1 C++ mode "ESC C-q" and TAB indenting issue.

[Bob]

[-- Attachment #1: Type: text/plain, Size: 4226 bytes --]

Alan,

>> I am now attempting to try your c-state-cache-init and find this is not
>> defined.
>
>That's exactly
>
>    M-: (c-state-cache-init)

Ahh, typed it in wrong.  I have been saving the buffer where the "
SelectInternalConvertableToVectorType" snippet is with the problematic '{'
in column 0.

So I am now able to eval your "(c-state-cache-init)" in the buffer.

The result is no difference, '{' still appears in column 0 when I TAB.
*Messages*
contains "1 (#o1, #x1)" after running it, if that means anything to you.

With regard to being more specific about what I type, that's going to be
really difficult.
Typically, I do a lot of kill - yanks, keyboard entry, and often return to
text to post-insert missing ','s or '>'s and then TAB or C-q C-j.  That's
when I discover a bad indentation case. Sometimes, by TAB-ing on previous
lines then returning to the problematic line, the indentation corrects. I
have done what you did with the snippet also - put it into a separate file,
start a separate emacs session. Each time I do that, the indentation
corrects ('{' is not in col 0) and doesn't get out of whack as I fiddle in
it. Makes me think the problem occurs with more than 1 file visited.

I am preserving the emacs session with the bad '{' indentation (col 0). Is
there any way we can narrow the issue by probing the buffer, such as with
your (c-state-cache-init) ?

Attached is the "c-submit-bug-report" run from the problematic buffer.

Bob

-----Original Message-----
From: Alan Mackenzie [mailto:acm@muc.de] 
Sent: Tuesday, February 16, 2010 8:39 AM
To: Bob
Cc: 'Chong Yidong'; 5570@debbugs.gnu.org
Subject: Re: bug#5570: Emacs 23.1.91.1 C++ mode "ESC C-q" and TAB indenting
issue.

Hi, Bob,

On Mon, Feb 15, 2010 at 09:14:52AM -0800, Bob wrote:
> Alan,

> I am now running two emacs 23.1.91.1, one I started last week with many
> files open in which I "work" etc., and one I just started with dribbling
> "on" to edit a file I have seen the issue in. After half-hours editing in
> this one, nothing bad has happened. I temporarily jumped back to the
"work"
> session to add a few needed lines and wammy! the problem appeared after
> adding 7 lines:

Could you be a little bit more exact in exactly what you did here.  I
have just fired up emacs 23.1.91, typed in 

    namespace foo
    {

(in column 0), then typed in the the lines that you've identified as
causing the error:

>   template<class _DataType>
>   struct SelectInternalConvertableToVectorType
>   {
>     typedef typename DataTypeTrait<_DataType>::TAbout	TAbout;
>
>     typedef TL::IF<TL::MemberOf<TAbout,EnumDTP::Internal>::value>
> TInternal;
>     typedef TL::IF<
> 	TL::MemberOf<TAbout,EnumDTP::HasVectorType>::value
> 	>					   TConvertableToVectorType;
> 	
>     typedef TL::IF<
>       (TInternal::value &&
> TConvertableToVectorType::value),_DataType,TL::False
>       >::Result						Result;
> };   <------------------------ Appears in column 1

However, that "};" for me appeared in the correct column, not in column
0.

> I wrote the buffer, started a 3rd emacs, went to the line, hit TAB and the
> indentation
> corrected:

That, at least, is refreshing.

>   };   <------------------------ Correct.

> Back in the original session TAB still places the '{' in column 1.

It doesn't for me.  But I _have_ seen this error happening.

So, could you please try and give me a precise recipe for how to
reproduce the error, down at the level of what the buffer should look
like before inserting these lines, and which keys to press to insert
them.  Was I correct in guessing that you typed in the lines when you
said you "added" them, as opposed to, say, C-y?

> I am now attempting to try your c-state-cache-init and find this is not
> defined.

That's exactly

    M-: (c-state-cache-init)

, including the parens round the word.  Are you certain you typed it
correctly?  It should be there in Emacs 23.1.91.

> Please advise what to do. I've attached "describe-bindings" if that helps.


> Thanks,
> Bob

> PS "C-c C-b" runs "c-submit-bug-report" - is that what you expect?

Yes.  It produces a dump of CC Mode's config as part of a mail buffer.
Cut and paste it into your own mail client.

-- 
Alan Mackenzie (Nuremberg, Germany).

[-- Attachment #2: Mail --]
[-- Type: application/octet-stream, Size: 4540 bytes --]

To: bug-cc-mode@gnu.org
Subject: CC Mode 5.31.8 (C++/la); Incorrect indentation
From: Bob@CACHE.i-did-not-set--mail-host-address--so-tickle-me
X-Reporter-Void-Vars-Found: auto-fill-mode (setq
 c-basic-offset 2
 c-comment-only-line-offset '(0 . 0)
 c-indent-comment-alist '((anchored-comment column . 0) (end-block space . 1)
			  (cpp-end-block space . 2))
 c-indent-comments-syntactically-p nil
 c-block-comment-prefix ""
 c-comment-prefix-regexp '((pike-mode . "//+!?\\|\\**") (awk-mode . "#+")
			   (other . "//+\\|\\**"))
 c-doc-comment-style '((java-mode . javadoc) (pike-mode . autodoc)
		       (c-mode . gtkdoc))
 c-cleanup-list '(scope-operator)
 c-hanging-braces-alist '((substatement-open before after)
			  (arglist-cont-nonempty))
 c-hanging-colons-alist nil
 c-hanging-semi&comma-criteria '(c-semi&comma-inside-parenlist)
 c-backslash-column 48
 c-backslash-max-column 72
 c-special-indent-hook '(c-gnu-impose-minimum)
 c-label-minimum-indentation 1
 c-offsets-alist '((inexpr-class . +)
		   (inexpr-statement . +)
		   (lambda-intro-cont . +)
		   (inlambda . c-lineup-inexpr-block)
		   (template-args-cont c-lineup-template-args +)
		   (incomposition . +)
		   (inmodule . +)
		   (innamespace . +)
		   (inextern-lang . +)
		   (composition-close . 0)
		   (module-close . 0)
		   (namespace-close . 0)
		   (extern-lang-close . 0)
		   (composition-open . 0)
		   (module-open . 0)
		   (namespace-open . 0)
		   (extern-lang-open . 0)
		   (objc-method-call-cont
		    c-lineup-ObjC-method-call-colons
		    c-lineup-ObjC-method-call
		    +
		    )
		   (objc-method-args-cont . c-lineup-ObjC-method-args)
		   (objc-method-intro . [0])
		   (friend . 0)
		   (cpp-define-intro c-lineup-cpp-define +)
		   (cpp-macro-cont . +)
		   (cpp-macro . [0])
		   (inclass . +)
		   (arglist-cont c-lineup-gcc-asm-reg 0)
		   (comment-intro
		    c-lineup-knr-region-comment
		    c-lineup-comment
		    )
		   (catch-clause . 0)
		   (else-clause . 0)
		   (do-while-closure . 0)
		   (access-label . -)
		   (case-label . 0)
		   (substatement . +)
		   (statement-case-intro . +)
		   (statement . 0)
		   (brace-entry-open . 0)
		   (brace-list-entry . 0)
		   (brace-list-intro . +)
		   (brace-list-close . 0)
		   (block-close . 0)
		   (block-open . 0)
		   (inher-intro . +)
		   (member-init-cont . c-lineup-multi-inher)
		   (member-init-intro . +)
		   (topmost-intro . 0)
		   (knr-argdecl . 0)
		   (func-decl-cont . +)
		   (inline-close . 0)
		   (class-close . 0)
		   (class-open . 0)
		   (defun-block-intro . +)
		   (defun-close . 0)
		   (defun-open . 0)
		   (c . c-lineup-C-comments)
		   (string . c-lineup-dont-change)
		   (topmost-intro-cont
		    first
		    c-lineup-topmost-intro-cont
		    c-lineup-gnu-DEFUN-intro-cont
		    )
		   (brace-list-open . +)
		   (inline-open . 0)
		   (arglist-close . c-lineup-arglist)
		   (arglist-intro . c-lineup-arglist-intro-after-paren)
		   (statement-case-open . +)
		   (label . 0)
		   (substatement-label . 0)
		   (knr-argdecl-intro . 5)
		   (statement-block-intro . +)
		   (statement-cont . 0)
		   (arglist-cont-nonempty . 0)
		   (stream-op . 0)
		   (inher-cont . 0)
		   (substatement-open . 0)
		   )
 c-buffer-is-cc-mode 'c++-mode
 c-tab-always-indent nil
 c-syntactic-indentation t
 c-syntactic-indentation-in-macros t
 c-ignore-auto-fill '(string cpp code)
 c-auto-align-backslashes t
 c-backspace-function 'backward-delete-char-untabify
 c-delete-function 'delete-char
 c-electric-pound-behavior nil
 c-default-style '((java-mode . "java") (awk-mode . "awk") (other . "gnu"))
 c-enable-xemacs-performance-kludge-p nil
 c-old-style-variable-behavior nil
 defun-prompt-regexp nil
 tab-width 8
 comment-column 32
 parse-sexp-ignore-comments t
 parse-sexp-lookup-properties t
 auto-fill-function nil
 comment-multi-line t
 comment-start-skip "\\(//+\\|/\\*+\\)\\s *"
 fill-prefix nil
 fill-column 70
 paragraph-start "[ 	]*\\(//+\\|\\**\\)[ 	]*$\\|^\f"
 adaptive-fill-mode t
 adaptive-fill-regexp "[ 	]*\\(//+\\|\\**\\)[ 	]*\\([ 	]*\\([-!|#%;>*·•‣⁃◦]+[ 	]*\\)*\\)"
 )

--text follows this line--


Emacs  : GNU Emacs 23.1.91.1 (i386-mingw-nt6.0.6002)
 of 2010-01-02 on PRETEST
Package: CC Mode 5.31.8 (C++/la)
Buffer Style: gnu
c-emacs-features: (pps-extended-state col-0-paren posix-char-classes gen-string-delim gen-comment-delim syntax-properties 1-bit)

current state:
==============

bug#5570: Emacs 23.1.91.1 C++ mode "ESC C-q" and TAB indenting issue.

[Alan Mackenzie]

Hi, Bob,

On Tue, Feb 16, 2010 at 10:35:37AM -0800, Bob wrote:
> Alan,

> >> I am now attempting to try your c-state-cache-init and find this is not
> >> defined.

> >That's exactly

> >    M-: (c-state-cache-init)

> Ahh, typed it in wrong.  I have been saving the buffer where the "
> SelectInternalConvertableToVectorType" snippet is with the problematic '{'
> in column 0.

> So I am now able to eval your "(c-state-cache-init)" in the buffer.

> The result is no difference, '{' still appears in column 0 when I TAB.
> *Messages*
> contains "1 (#o1, #x1)" after running it, if that means anything to you.

Yes, thanks.  The "1 .." is just random noise.  The fact that
`c-state-cache-init' fixes nothing tells me that the bug's not to do with
a massive overhaul of the "state cache" (a cache of brace/paren/bracket
positions) I committed several months ago.

> With regard to being more specific about what I type, that's going to be
> really difficult.

OK, I understand that.

> Typically, I do a lot of kill - yanks, keyboard entry, and often return
> to text to post-insert missing ','s or '>'s and then TAB or C-q C-j.
> That's when I discover a bad indentation case. Sometimes, by TAB-ing on
> previous lines then returning to the problematic line, the indentation
> corrects. I have done what you did with the snippet also - put it into
> a separate file, start a separate emacs session. Each time I do that,
> the indentation corrects ('{' is not in col 0) and doesn't get out of
> whack as I fiddle in it. Makes me think the problem occurs with more
> than 1 file visited.

I have a hyptothesis: noramlly, when you type in a < or > which "looks
like" a template delimiter, CC Mode marks it as an openining/closing
parenthesis.  These can, and do, get confused with "proper" parens, { } (
) [ ], sometimes.  It seems likely that on certain deletions, these
marked <s and >s become unbalanced, and this confuses the indentation.

I committed a fix for this on 2010-02-04, barely 2 weeks ago.  This fix
ensures that these marks only get applied to < and > in balanced pairs,
and when stuff is deleted does the Right Thing.  The fix hasn't yet found
its way into a pretest.  If you test on the bzr head, could you try it
out, please.  Otherwise, please be patient and try it out on the NEXT
pretest version, when that gets released.

> I am preserving the emacs session with the bad '{' indentation (col 0).
> Is there any way we can narrow the issue by probing the buffer, such as
> with your (c-state-cache-init) ?

Could you perhaps test the above hypothesis, please?  Turn on
`show-paren-mode'.  When you next hit this bad state in a buffer, place
point on a < or >, and see if it mismatches with a proper paren/brace.

> Attached is the "c-submit-bug-report" run from the problematic buffer.

Much appreciated.

> Bob

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#5570: Emacs 23.1.91.1 C++ mode "ESC C-q" and TAB indenting issue.

[Bob]

[-- Attachment #1: Type: text/plain, Size: 1119 bytes --]

Alan,

Attached is "Foo.cpp" and "dribble" (if I did it correctly, dribble recorded
what I typed).

It illustrates a case where indentation is one level deeper than it should
be. (The other case we've been working on is one level shallower than it
should be). Maybe they are related?

visit-file "Foo.cpp", note indentation for "class Bar" is too deep. Now, TAB
on the "class Bar" line the indentation corrects. If you reconstruct the
file from "dribble" you should see TAB on the "class Bar" line incorrectly
indents too deep.

My ".emacs" file sets indentation to 2 spaces. I would guess there is an
issue where
the default (I think 4 spaces) is being used instead of my setting.

I've also attached my ".emacs" file as "emacs_init" (Outlook doesn't attach
files without a file-name). Please excuse the accumulation of junk in it
from many years of emacs use and
recommendations obtained from internet sites.

Bob

Foo.cpp:
----------------------------------------
template<
    class Bar,
    int _depth = -1
    >
class Foo;

template<
  int _depth = -1
  >
class Foo
{
};
-----------------------------------------

[-- Attachment #2: dribble --]
[-- Type: application/octet-stream, Size: 564 bytes --]

[-- Attachment #3: Foo.cpp --]
[-- Type: text/plain, Size: 115 bytes --]


template<
    class Bar,
    int _depth = -1
    >
class Foo;

template<
  int _depth = -1
  >
class Foo
{
};

  

[-- Attachment #4: emacs_init --]
[-- Type: application/octet-stream, Size: 7554 bytes --]

bug#5570: Emacs 23.1.91.1 C++ mode "ESC C-q" and TAB indenting issue.

[Alan Mackenzie]

Hi, Bob,

On Wed, Feb 17, 2010 at 08:40:36AM -0800, Bob wrote:
> Alan,

> Attached is "Foo.cpp" and "dribble" (if I did it correctly, dribble recorded
> what I typed).

> It illustrates a case where indentation is one level deeper than it should
> be. (The other case we've been working on is one level shallower than it
> should be). Maybe they are related?

I suspect not, but who knows?  :-(

However, I can reproduce this bug; the dribble file and your CC Mode
configuration file were utterly essential to my doing so.  Thanks!

> visit-file "Foo.cpp", note indentation for "class Bar" is too deep. Now, TAB
> on the "class Bar" line the indentation corrects. If you reconstruct the
> file from "dribble" you should see TAB on the "class Bar" line incorrectly
> indents too deep.

Yes.

> My ".emacs" file sets indentation to 2 spaces. I would guess there is an
> issue where
> the default (I think 4 spaces) is being used instead of my setting.

No, that's not it.  In the course of creating this file, as per your
dribble, some "text properties" (deep in the bowels of CC Mode) are being
spuriously applied to certain characters in the buffer.  I'll track down
exactly which keystroke provoked this, and hopefully be able to fix it
sometime soonish.

Thanks for taking all this trouble to help me reproduce the bug.

> Bob

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#5570: Emacs 23.1.91.1 C++ mode "ESC C-q" and TAB indenting issue.

[Bob]

Alan,

That's good news! Please let me know when you get it fixed and I'd like to
try it out.

BTW, the indenting and paren matching is a world better than 21.2, esp. with
regard to template '<' and '>' characters. Good job!

Bob

-----Original Message-----
From: Alan Mackenzie [mailto:acm@muc.de] 
Sent: Thursday, February 18, 2010 7:13 AM
To: Bob
Cc: 'Chong Yidong'; 5570@debbugs.gnu.org
Subject: Re: bug#5570: Emacs 23.1.91.1 C++ mode "ESC C-q" and TAB indenting
issue.

Hi, Bob,

On Wed, Feb 17, 2010 at 08:40:36AM -0800, Bob wrote:
> Alan,

> Attached is "Foo.cpp" and "dribble" (if I did it correctly, dribble
recorded
> what I typed).

> It illustrates a case where indentation is one level deeper than it should
> be. (The other case we've been working on is one level shallower than it
> should be). Maybe they are related?

I suspect not, but who knows?  :-(

However, I can reproduce this bug; the dribble file and your CC Mode
configuration file were utterly essential to my doing so.  Thanks!

> visit-file "Foo.cpp", note indentation for "class Bar" is too deep. Now,
TAB
> on the "class Bar" line the indentation corrects. If you reconstruct the
> file from "dribble" you should see TAB on the "class Bar" line incorrectly
> indents too deep.

Yes.

> My ".emacs" file sets indentation to 2 spaces. I would guess there is an
> issue where
> the default (I think 4 spaces) is being used instead of my setting.

No, that's not it.  In the course of creating this file, as per your
dribble, some "text properties" (deep in the bowels of CC Mode) are being
spuriously applied to certain characters in the buffer.  I'll track down
exactly which keystroke provoked this, and hopefully be able to fix it
sometime soonish.

Thanks for taking all this trouble to help me reproduce the bug.

> Bob

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#5570: Emacs 23.1.91.1 C++ mode "ESC C-q" and TAB indenting issue.

[Alan Mackenzie]

Hi, Bob,

I think I've got the problem nailed.  And I now think you were right
when you suggested it's related to the other indentation problems.

Would you please apply this patch to cc-mode.el, and tell me if the
problem's been fixed.

Thanks!


*** orig/cc-mode.el	2010-02-18 16:29:49.000000000 +0000
--- cc-mode.el	2010-02-19 20:15:14.630249504 +0000
***************
*** 635,641 ****
      (save-excursion
        (if c-get-state-before-change-functions
  	  (mapc (lambda (fn)
! 		  (funcall fn beg end))
  		c-get-state-before-change-functions))
        (if c-before-font-lock-function
  	  (funcall c-before-font-lock-function (point-min) (point-max)
--- 635,641 ----
      (save-excursion
        (if c-get-state-before-change-functions
  	  (mapc (lambda (fn)
! 		  (funcall fn (point-min) (point-max)))
  		c-get-state-before-change-functions))
        (if c-before-font-lock-function
  	  (funcall c-before-font-lock-function (point-min) (point-max)
***************
*** 1031,1036 ****
--- 1031,1044 ----
  	  (when (> beg end)
  	    (setq beg end)))
  
+ 	;; C-y is capable of spuriously converting category properties
+ 	;; c-</>-as-paren-syntax into hard syntax-table properties.  Remove
+ 	;; these when it happens.
+ 	(c-clear-char-property-with-value beg end 'syntax-table
+ 					  c-<-as-paren-syntax)
+ 	(c-clear-char-property-with-value beg end 'syntax-table
+ 					  c->-as-paren-syntax)
+ 
  	(c-trim-found-types beg end old-len) ; maybe we don't need all of these.
  	(c-invalidate-sws-region-after beg end)
  	(c-invalidate-state-cache beg)

On Thu, Feb 18, 2010 at 07:53:34AM -0800, Bob wrote:
> Alan,
> 
> That's good news! Please let me know when you get it fixed and I'd like to
> try it out.
> 
> BTW, the indenting and paren matching is a world better than 21.2, esp. with
> regard to template '<' and '>' characters. Good job!
> 
> Bob

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#5570: Emacs 23.1.91.1 C++ mode "ESC C-q" and TAB indenting issue.

[Paul Pogonyshev]

Severity: important

> Has anyone else seen this issue?

I see it so often it is not fun.  I'm currently compiling an older
Emacs version because indentation's becoming broken every second line
is unbearable.

Paul

bug#5570: Emacs 23.1.91.1 C++ mode "ESC C-q" and TAB indenting issue.

[Paul Pogonyshev]

Chong Yidong wrote:
> Paul Pogonyshev <pogonyshev@gmx.net> writes:
> 
> > Chong Yidong wrote:
> >> Paul Pogonyshev <pogonyshev@gmx.net> writes:
> >> 
> >> >> Any chance you might try the head versions in the bzr repository?
> >> >
> >> > OK, but a bit later.  I guess it is of little help, but currently I
> >> > use revision 98816 (just chose one at random) and cc-mode works fine.
> >> 
> >> It would be nice if you could test the patch Alan posted at
> >> 
> >>   http://debbugs.gnu.org/cgi/bugreport.cgi?bug=5570#40
> >> 
> >> since "Bob" has not responded to the request to test it.
> >
> > It doesn't apply to trunk and I'd prefer not to guess how to merge.
> > Can you rediff it?
> 
> Here, try this: [...]

Thanks, indentation problems seem to be gone.  Or at least don't
happen nearly that often.

Paul

bug#5570: Emacs 23.1.91.1 C++ mode "ESC C-q" and TAB indenting issue.

[Paul Pogonyshev]

I wrote:
> Thanks, indentation problems seem to be gone.  Or at least don't
> happen nearly that often.

Still happens, though much more rarely than before.  No sure way to
reproduce, but in this short source code file:

#include "engine/strategy/evaluation/material.hpp"

#include <utility>


using namespace engine;

using namespace std;


int
MaterialEvaluationFunction::do_evaluate_material (Board::PackedMaterialData material)
{
  const int  score            = _algorithm->evaluate (material);
  const int  normalized_score = ((score * get_first_rabbit_value ())
                                 / _algorithm->get_first_rabbit_value ());

_cache.insert (make_pair (material, normalized_score));
}

I'm getting wrongly indented '_cache.insert ...' line.  C-c C-s on it
also shows 'topmost-intro'.  I guess this heavily depends on the order
in which code is typed.

After I killed the buffer and reopened it, indentation is back to
normal and C-c C-s on that line correctly shows 'statement'.

Paul

bug#5570: automatic cache validation

[Paul Pogonyshev]

After March the 1st changes indentation in C++ and related modes is
better, but it still breaks often enough --- to the tune of once every
3--5 minutes or so.  Both when editing C++ and Java code.

As I understand, the issue is that some cached information becomes
wrong after certain (undetermined) buffer changes.  Maybe it is better
to write an automated cache validation code rather than search for a
certain way to reproduce stuff?

E.g. just build information from scratch and assert that cached
information is the same.  Ideally, such checks should be performed
after every change, so that you'd be able to find which change exactly
trigerred a bug.  (Granted, this would make debugging version slow as
hell, but with this rate of error occurence it's not like you'd need
to run it for days.)

Without automated error detection tools cc-mode is bound to have
similar issues after every second non-trivial change.

Paul

bug#5570: automatic cache validation

[Chong Yidong]

Paul Pogonyshev <pogonyshev@gmx.net> writes:

> After March the 1st changes indentation in C++ and related modes is
> better, but it still breaks often enough --- to the tune of once every
> 3--5 minutes or so.  Both when editing C++ and Java code.

This is troubling.

If the CC-mode changes are still not quite stabilized, I would like to
consider postphoning them to post-23.2.  Alan, what's your opinion?
Could we revert to the version in the branch prior to 2010-02-04 (which
IIUC is the change that introduced these problems), or do you consider
those changes too important for the release?

bug#5570: automatic cache validation

[Alan Mackenzie]

Hi, Paul,

On Thu, Mar 18, 2010 at 10:05:11PM +0200, Paul Pogonyshev wrote:
> After March the 1st changes indentation in C++ and related modes is
> better, but it still breaks often enough --- to the tune of once every
> 3--5 minutes or so.  Both when editing C++ and Java code.

How about plain C?  Is the breakage connected with templates/generics, as
far as you can see?

What is the breakage like?  Is it a one-off failure which repairs itself
after a few more edits, or is it more like a creeping death?  Does
reinitializing the mode (M-x c++-mode) repair the indentation?  Does
saving the file and reloading it fix things?

Does
    M-x (c-state-cache-init)
fix it?

> As I understand, the issue is that some cached information becomes
> wrong after certain (undetermined) buffer changes.

This is very likely.

> Maybe it is better to write an automated cache validation code rather
> than search for a certain way to reproduce stuff?

For which cache, though?  Hence the questions above.  

> E.g. just build information from scratch and assert that cached
> information is the same.  Ideally, such checks should be performed
> after every change, so that you'd be able to find which change exactly
> trigerred a bug.  (Granted, this would make debugging version slow as
> hell, but with this rate of error occurence it's not like you'd need
> to run it for days.)

I should be able to get this done by, say, Monday for the "state-cache".
Again, does M-x (c-state-cache-init) clear the fault?

> Without automated error detection tools cc-mode is bound to have
> similar issues after every second non-trivial change.

I strongly believe it's a very small finite number of distinct bugs
causing the problems.  Anything more you can tell me about how it breaks
would be of the utmost help.

> Paul

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#5570: automatic cache validation

[Jan Djärv]

Alan Mackenzie skrev 2010-03-19 22.56:
> Hi, Paul,
>
> On Thu, Mar 18, 2010 at 10:05:11PM +0200, Paul Pogonyshev wrote:
>> After March the 1st changes indentation in C++ and related modes is
>> better, but it still breaks often enough --- to the tune of once every
>> 3--5 minutes or so.  Both when editing C++ and Java code.
>
> How about plain C?  Is the breakage connected with templates/generics, as
> far as you can see?

FWIW, I see breakage in plain C also.

>
> What is the breakage like?  Is it a one-off failure which repairs itself
> after a few more edits, or is it more like a creeping death?  Does
> reinitializing the mode (M-x c++-mode) repair the indentation?  Does
> saving the file and reloading it fix things?

I now more or less automatically do M-x revert buffer when this happens.

	Jan D.

bug#5570: automatic cache validation

[Alan Mackenzie]

Hi, Jan,

On Fri, Mar 19, 2010 at 11:45:40PM +0100, Jan Djärv wrote:


> Alan Mackenzie skrev 2010-03-19 22.56:
> >Hi, Paul,

> >On Thu, Mar 18, 2010 at 10:05:11PM +0200, Paul Pogonyshev wrote:
> >>After March the 1st changes indentation in C++ and related modes is
> >>better, but it still breaks often enough --- to the tune of once every
> >>3--5 minutes or so.  Both when editing C++ and Java code.

> >How about plain C?  Is the breakage connected with templates/generics, as
> >far as you can see?

> FWIW, I see breakage in plain C also.

That's worth a great deal to know.  It suggests that the changes of
2010-02-04 aren't responsble for this bug.

> >What is the breakage like?  Is it a one-off failure which repairs itself
> >after a few more edits, or is it more like a creeping death?  Does
> >reinitializing the mode (M-x c++-mode) repair the indentation?  Does
> >saving the file and reloading it fix things?

> I now more or less automatically do M-x revert buffer when this happens.

When _what_ happens?  I take it, then, that revert-buffer fixes the bad
state, and you save the file first before reverting it.  Any chance you
could answer the above questions, together with the further one:

Does
    M-x (c-state-cache-init)
fix it?
    
These answers would be of the utmost help.  Thanks!

> 	Jan D.

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#5570: automatic cache validation

[Jan D.]

Alan Mackenzie wrote:
> Hi, Jan,
> 
> On Fri, Mar 19, 2010 at 11:45:40PM +0100, Jan Djärv wrote:
> 
> 
>> Alan Mackenzie skrev 2010-03-19 22.56:
>>> Hi, Paul,
> 
>>> On Thu, Mar 18, 2010 at 10:05:11PM +0200, Paul Pogonyshev wrote:
>>>> After March the 1st changes indentation in C++ and related modes is
>>>> better, but it still breaks often enough --- to the tune of once every
>>>> 3--5 minutes or so.  Both when editing C++ and Java code.
> 
>>> How about plain C?  Is the breakage connected with templates/generics, as
>>> far as you can see?
> 
>> FWIW, I see breakage in plain C also.
> 
> That's worth a great deal to know.  It suggests that the changes of
> 2010-02-04 aren't responsble for this bug.
> 
>>> What is the breakage like?  Is it a one-off failure which repairs itself
>>> after a few more edits, or is it more like a creeping death?  Does
>>> reinitializing the mode (M-x c++-mode) repair the indentation?  Does
>>> saving the file and reloading it fix things?
> 
>> I now more or less automatically do M-x revert buffer when this happens.
> 
> When _what_ happens? 

When C-mode thinks everything is topmost-intro.

> I take it, then, that revert-buffer fixes the bad
> state, and you save the file first before reverting it.  Any chance you
> could answer the above questions, together with the further one:
> 
> Does
>     M-x (c-state-cache-init)
> fix it?
>     

No, it does not.

	Jan D.

bug#5570: automatic cache validation

[Alan Mackenzie]

Hi, Jan

On Sat, Mar 20, 2010 at 03:49:07PM +0100, Jan D. wrote:
> Alan Mackenzie wrote:
> >On Fri, Mar 19, 2010 at 11:45:40PM +0100, Jan Djärv wrote:

> >>>How about plain C?  Is the breakage connected with
> >>>templates/generics, as far as you can see?

> >>FWIW, I see breakage in plain C also.

> >That's worth a great deal to know.  It suggests that the changes of
> >2010-02-04 aren't responsble for this bug.

> >>I now more or less automatically do M-x revert buffer when this happens.

> >When _what_ happens? 

> When C-mode thinks everything is topmost-intro.

OK.  I get the impression this is VERY annoying, but that it happens very
frequently rather than just now and then.  Good!

> >I take it, then, that revert-buffer fixes the bad
> >state, and you save the file first before reverting it.  Any chance you
> >could answer the above questions, together with the further one:

> >Does
> >    M-x (c-state-cache-init)
> >fix it?


> No, it does not.

I'm assuming your Emacs is pretty much the newest bzr version.

Would you please choose a C file, as small as will work, and start an
Emacs session with a dribble file (see page "Recording Input" in the
Elisp manual).  Edit the C file until indentation goes wrong.  Then send
me the dribble file, the (original) C file, and your CC Mode
configuration (generated by C-c C-b in the C file).

Apologies for this horrible bug, and thanks for you help in nailing it
down!

> 	Jan D.

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#5570: automatic cache validation

[Jan Djärv]

Alan Mackenzie skrev 2010-03-21 12.16:

>
> I'm assuming your Emacs is pretty much the newest bzr version.

Pretty much.

>
> Would you please choose a C file, as small as will work, and start an
> Emacs session with a dribble file (see page "Recording Input" in the
> Elisp manual).  Edit the C file until indentation goes wrong.

It isn't that easy to reproduce, it just sort of happens.
I have no idea if it will happen on a small file.

> Then send
> me the dribble file, the (original) C file, and your CC Mode
> configuration (generated by C-c C-b in the C file).
>

I'll try.

	Jan D.

bug#5570: automatic cache validation

[Jan Djärv]

Alan Mackenzie skrev:
> Hi, Jan
> 
> 
>>> Does
>>>    M-x (c-state-cache-init)
>>> fix it?
> 
> 
>> No, it does not.
> 

I just got the problem again, while editing gtkutil.c in Emacs.  There
M-: (c-state-cache-init) did indeed fix the problem.

So it sometimes help.  I guess this only adds to the confusion...

	Jan D.

bug#5570: automatic cache validation

[Paul Pogonyshev]

Jan Djärv wrote:
> Alan Mackenzie skrev:
> >>> Does
> >>>    M-x (c-state-cache-init)
> >>> fix it?
> > 
> > 
> >> No, it does not.
> > 
> 
> I just got the problem again, while editing gtkutil.c in Emacs.  There
> M-: (c-state-cache-init) did indeed fix the problem.
> 
> So it sometimes help.  I guess this only adds to the confusion...

I guess that's because there are several similar bugs.  At the rate
fixing goes (or rather doesn't go) I think I'll compile and use an old
Emacs without indentation bugs again.

Paul


P.S.  By the way: maybe this change did fix the original bug, but it
didn't improve overall indentation problems in CC modes noticeably.

2010-03-24  Alan Mackenzie  <acm@muc.de>

	* progmodes/cc-engine.el (c-remove-stale-state-cache):
	Fix off-by-one error.  Fixes bug #5747.

bug#5570: automatic cache validation

[Chong Yidong]

> >>> Does
> >>>    M-x (c-state-cache-init)
> >>> fix it?
> >
> >> No, it does not.
> >
>
> I just got the problem again, while editing gtkutil.c in Emacs.  There
> M-: (c-state-cache-init) did indeed fix the problem.
>
> So it sometimes help.  I guess this only adds to the confusion...

You sent this message on March 22.  That was the date Alan checked this
into the branch:

   2010-03-22  Alan Mackenzie  <acm@muc.de>

      * progmodes/cc-engine.el (c-remove-stale-state-cache):
      Fix off-by-one error.  Fixes bug #5747.

The patch was merged to the trunk a couple days later.  I don't know
whether you were running with it when you made the above observation.

So, are you still observing the problem after the patch?

bug#5570: automatic cache validation

[Chong Yidong]

Hi Alan,

I found a small recipe for reproducing a (the?) cache bug on the trunk:

1. emacs -Q

2. C-x C-f foo.c RET

3. Type the following (from view-lossage):

  i n t SPC m a i n SPC ( ) <return>
  { <return> f o o ; <return> } M-< <return>
  <up> # i n c l u d e SPC < s t d i o . h > <return>
  <down> <down> <down> <tab>

At this point, the line "foo;" becomes indented to the zeroth column.

Does this help?

bug#5570: Emacs 23.1.91.1 C++ mode "ESC C-q" and TAB indenting issue.

[Lars Ingebrigtsen]

"Bob" <purchasebyemail1@comcast.net> writes:

> As I work with text in C++ mode the indentation begins to deteriorate. For
> example, TAB on the class line wrongly indents it:
>
>     template<class _Enum>
>
>       class Build<TL::End,_Enum>

There were several test cases in these merged bug reports, and I think I
tried them all now, and none of them reproduce for me.

So I'm guessing that Alan's changes in this bug report, or to cc-mode in
the ten years that have passed, have fixed all the reported bugs here,
so I'm closing this bug report.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no