bug#1756: awk-mode: An empty line is not a paragraph separator (should be)

bug#1756: awk-mode: An empty line is not a paragraph separator (should be)

[Teemu Likonen]

In awk-mode an empty line is not considered a paragraph separator, only
lines containing whitespace and "#" character(s) are. After running the
command "M-x awk-mode" the value of both paragraph-start and
paragraph-separate are set as follows:

    "[ \t]*\\(#+\\)[ \t]*$\\|^\f"
              ^^

See, at least one # is required. I think better default would be #* so
that empty lines or lines with only whitespace would be paragraph
separators too. I suggest doing the following change:


diff --git i/lisp/progmodes/cc-vars.el w/lisp/progmodes/cc-vars.el
index 18f314c..62c85f0 100644
--- i/lisp/progmodes/cc-vars.el
+++ w/lisp/progmodes/cc-vars.el
@@ -483,7 +483,7 @@ style comments."
 
 (defcustom-c-stylevar c-comment-prefix-regexp
   '((pike-mode . "//+!?\\|\\**")
-    (awk-mode . "#+")
+    (awk-mode . "#*")
     (other . "//+\\|\\**"))
   "*Regexp to match the line prefix inside comments.
 This regexp is used to recognize the fill prefix inside comments for


(pike-mode and "other" have a comment prefix regexp which matches a zero
number of comment prefix characters: \**)


There is another and related bug. It is possible to configure the
comment prefix regexp with the option c-comment-prefix-regexp. But the
problem is that the option only takes effect when awk-mode is turned on.
If user later changes indentation style with the command c-set-style
(bound to C-c .) then it seems that the hard-coded default (#+) takes
preference over user's c-comment-prefix-regexp settings.

Re: bug#1756: awk-mode: An empty line is not a paragraph separator (should be)

[Alan Mackenzie]

Hi, Teemu!

On Thu, Jan 01, 2009 at 10:27:17AM +0200, Teemu Likonen wrote:
> In awk-mode an empty line is not considered a paragraph separator, only
> lines containing whitespace and "#" character(s) are. After running the
> command "M-x awk-mode" the value of both paragraph-start and
> paragraph-separate are set as follows:

>     "[ \t]*\\(#+\\)[ \t]*$\\|^\f"
>               ^^

> See, at least one # is required. I think better default would be #* so
> that empty lines or lines with only whitespace would be paragraph
> separators too.

Agreed.  Thanks for the report.

> I suggest doing the following change: 
> [ Tweaking the definition of c-comment-prefix-regexp. ]

I'd rather not do it this way, because although c-comment-prefix-regexp
isn't currently used anywhere else, it might be in the future.  A blank
line cannot be a comment prefix in AWK.

> (pike-mode and "other" have a comment prefix regexp which matches a zero
> number of comment prefix characters: \**)

Can you please try the following patch (it might be a few lines offset
from the given line numbers).  It checks whether the comment prefix
matches a blank line, and fixes paragraph-s{tart,eparate} if it doesn't.

#########################################################################

*** orig/cc-styles.el	2009-01-05 11:49:10.753657960 +0000
--- cc-styles.el	2009-01-05 15:29:34.569332408 +0000
***************
*** 512,525 ****
  			  (assoc 'other c-comment-prefix-regexp)))
  	  c-comment-prefix-regexp))
  
!   (let ((comment-line-prefix
! 	 (concat "[ \t]*\\(" c-current-comment-prefix "\\)[ \t]*")))
  
!     (setq paragraph-start (concat comment-line-prefix
  				  c-paragraph-start
  				  "\\|"
  				  page-delimiter)
! 	  paragraph-separate (concat comment-line-prefix
  				     c-paragraph-separate
  				     "\\|"
  				     page-delimiter)
--- 512,532 ----
  			  (assoc 'other c-comment-prefix-regexp)))
  	  c-comment-prefix-regexp))
  
!   (let* ((empty-is-prefix (string-match c-current-comment-prefix ""))
! 	 (nonws-comment-line-prefix
! 	  (concat "\\(" c-current-comment-prefix "\\)[ \t]*"))
! 	 (comment-line-prefix (concat "[ \t]*" nonws-comment-line-prefix))
! 	 (blank-or-comment-line-prefix
! 	  (concat "[ \t]*"
! 		  (if empty-is-prefix "" "\\(")
! 		  nonws-comment-line-prefix
! 		  (if empty-is-prefix "" "\\)?"))))
  
!     (setq paragraph-start (concat blank-or-comment-line-prefix
  				  c-paragraph-start
  				  "\\|"
  				  page-delimiter)
! 	  paragraph-separate (concat blank-or-comment-line-prefix
  				     c-paragraph-separate
  				     "\\|"
  				     page-delimiter)

#########################################################################

> There is another and related bug. It is possible to configure the
> comment prefix regexp with the option c-comment-prefix-regexp. But the
> problem is that the option only takes effect when awk-mode is turned on.
> If user later changes indentation style with the command c-set-style
> (bound to C-c .) then it seems that the hard-coded default (#+) takes
> preference over user's c-comment-prefix-regexp settings.

I'll get back to you on this one.

Thanks again for the bug report.

-- 
Alan Mackenzie (Nuremberg, Germany).

------------------------------------------------------------------------------

bug#1756: awk-mode: An empty line is not a paragraph separator (should be)

[Teemu Likonen]

Alan Mackenzie (2009-01-05 16:12 +0000) wrote:

> Can you please try the following patch (it might be a few lines offset
> from the given line numbers). It checks whether the comment prefix
> matches a blank line, and fixes paragraph-s{tart,eparate} if it
> doesn't.

I tried it and it seems to work nicely. I'm not competent enough to
critically review the actual code changes you introduced but, well, so
far I think it works. :-)

bug#1756: awk-mode: An empty line is not a paragraph separator (should be)

[Alan Mackenzie]

Hi again, Teemu!

On Thu, Jan 01, 2009 at 10:27:17AM +0200, Teemu Likonen wrote:

[ .... ]

> There is another and related bug. It is possible to configure the
> comment prefix regexp with the option c-comment-prefix-regexp. But the
> problem is that the option only takes effect when awk-mode is turned
> on.  If user later changes indentation style with the command
> c-set-style (bound to C-c .) then it seems that the hard-coded default
> (#+) takes preference over user's c-comment-prefix-regexp settings.

Can you give me precise recipe to reproduce this, please.  It worked OK
for me when I tried it.  When I did the following:

(i) M-: (setcdr (cadr c-comment-prefix-regexp) "#*") ; Changes the entry
          ; from (awk . "#+") to (awk . "#*")
(ii) C-c . <CR> awk  ; from within an AWK buffer

, M-: paragraph-start gave me "[ \t]*\\(#*\\)[ \t]*$\\|^\f"
                                        ^^
which has incorporated the new value from c-comment-prefix-regexp.

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#1756: awk-mode: An empty line is not a paragraph separator (should be)

[Teemu Likonen]

Alan Mackenzie (2009-01-05 18:35 +0000) wrote:

> Can you give me precise recipe to reproduce this, please.  It worked OK
> for me when I tried it.  When I did the following:
>
> (i) M-: (setcdr (cadr c-comment-prefix-regexp) "#*") ; Changes the entry
>           ; from (awk . "#+") to (awk . "#*")
> (ii) C-c . <CR> awk  ; from within an AWK buffer
>
> , M-: paragraph-start gave me "[ \t]*\\(#*\\)[ \t]*$\\|^\f"
>                                         ^^
> which has incorporated the new value from c-comment-prefix-regexp.

I can reproduce it by setting c-comment-prefix-regexp through the
customize system:

    M-x customize-option RET c-comment-prefix-regexp RET

Select "Override style settings" and "Mode-specific regexps". Type #* to
the AWK string and save the settings. Now, when I do "M-x awk-mode" I
get this paragraph-start:

    "[ \t]*\\(#*\\)[ \t]*$\\|^\f"

After "C-c . awk RET" it changes to this:

    "[ \t]*\\(\\(#+\\)[ \t]*\\)?$\\|^\f"

Even though I chose to override the style settings "#*" changes to "#+".

I have your previous patch applied so there is this additional
subexpression level "\(...\)?" too.

bug#1756: awk-mode: An empty line is not a paragraph separator (should be)

[Alan Mackenzie]

Hi, Teemu!

On Mon, Jan 05, 2009 at 08:38:05PM +0200, Teemu Likonen wrote:
> Alan Mackenzie (2009-01-05 18:35 +0000) wrote:

> > Can you give me precise recipe to reproduce this, please.  It worked OK
> > for me when I tried it.  When I did the following:
> >
> > (i) M-: (setcdr (cadr c-comment-prefix-regexp) "#*") ; Changes the entry
> >           ; from (awk . "#+") to (awk . "#*")
> > (ii) C-c . <CR> awk  ; from within an AWK buffer
>
> I have your previous patch applied so there is this additional
> subexpression level "\(...\)?" too. >
> > , M-: paragraph-start gave me "[ \t]*\\(#*\\)[ \t]*$\\|^\f"
> >                                         ^^
> > which has incorporated the new value from c-comment-prefix-regexp.

> I can reproduce it by setting c-comment-prefix-regexp through the
> customize system:

>     M-x customize-option RET c-comment-prefix-regexp RET

> Select "Override style settings" and "Mode-specific regexps". Type #* to
> the AWK string and save the settings. Now, when I do "M-x awk-mode" I
> get this paragraph-start:

>     "[ \t]*\\(#*\\)[ \t]*$\\|^\f"
                ^^

Which is expected and correct - the awk bit of the global value of
c-comment-prefix-regexp, "#*", has been incorporated into
paragraph-start.

> After "C-c . awk RET" it changes to this:

>     "[ \t]*\\(\\(#+\\)[ \t]*\\)?$\\|^\f"
                   ^^

> Even though I chose to override the style settings "#*" changes to "#+".

I don't think this is a bug.  You asked for "awk" style to be set on the
buffer, and this is exactly what you got.  Customizing
c-comment-prefix-regexp has set the global value of this variable (which
was previously the symbol 'set-from-style); it hasn't changed the value
in any style.

I agree this is confusing, very confusing, and would very much like to
make it less confusing.  I am guessing that the cause is in the fine CC
Mode manual, page "Config Basics", in this bit:

    If you make conflicting settings in several of these ways, the way
    that takes precedence is the one that appears latest in this list:

        Style
        Top-level command or "customization interface"
        Hook
        File Style

This sentence only applies when initialising the mode, so perhaps it
would be better if amended something like this:

    When you initialize the buffer, the settings are made in the
    following order.  So if you make conflicting settings in several of
    these ways, the way that takes precedence is the one that appears
    latest in the list(2):

        Style
        Top-level command or "customization interface"
        Hook
        File Style

    ....
    ....

    ---------- Footnotes ----------
    (2) If you later call `c-set-style' (C-c .), all the style variables
    will get set to the style you select.
 
What do you think?

[ .... ]

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: bug#1756: awk-mode: An empty line is not a paragraph separator (should be)

[Teemu Likonen]

Alan Mackenzie (2009-01-06 16:15 +0000) wrote:

>> After "C-c . awk RET" it changes to this:
>
>>     "[ \t]*\\(\\(#+\\)[ \t]*\\)?$\\|^\f"
>                    ^^
>
>> Even though I chose to override the style settings "#*" changes to "#+".
>
> I don't think this is a bug.  You asked for "awk" style to be set on the
> buffer, and this is exactly what you got.

Well, I asked that, but I had also explicitly asked to override style
settings (through the customize interface). My opinion is that the
current behavior is so confusing that it's either a bug or simply wrong
design decision.

I don't mean to complain too loudly, though. See below.

> I am guessing that the cause is in the fine CC Mode manual, [...]

> [...] perhaps it would be better if amended something like this:
>
>     When you initialize the buffer, the settings are made in the
>     following order.  So if you make conflicting settings in several of
>     these ways, the way that takes precedence is the one that appears
>     latest in the list(2):
>
>         Style
>         Top-level command or "customization interface"
>         Hook
>         File Style

>     ---------- Footnotes ----------
>     (2) If you later call `c-set-style' (C-c .), all the style variables
>     will get set to the style you select.
>  
> What do you think?

Well, my real opinion first: User's explicit "override style settings"
setting should just do that: override style settings no matter what
style changes the user does while editing the buffer. For example, user
may want to choose certain custom comment prefix settings and, while
editing the code, she may want to experiment with different style
settings or convert the code from one style to another. Logically, if
she wants to get comment prefix settings from a pre-defined style she
chooses "Use style settings" (c-comment-prefix-regexp). If she wants to
use her own comment prefix she chooses "Override style settings". I
think this is the most intuitive behaviour. Currently user can only
override style settings for the initial style state.

But if you choose to maintain the current behaviour anyway, then I'd say
that the changes you suggested for the documentation are very good. I'd
also suggest to change the documentation of c-comment-prefix-regexp
variable so that it tells how the variable effects in the practice.

Anyway, the empty-line bug is the main thing. If that bug in Awk mode
gets fixed (you already did with that patch) then there shouldn't be
much need for changing comment prefixes in the first place. So I'm
really more on the "thank you" side than on "complaining" side. :-)

------------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It is the best place to buy or sell services for
just about anything Open Source.
http://p.sf.net/sfu/Xq1LFB

Re: bug#1756: awk-mode: An empty line is not a paragraph separator (should be)

[Alan Mackenzie]

Hi, Teemu,

On Wed, Jan 07, 2009 at 06:32:56PM +0200, Teemu Likonen wrote:
> Alan Mackenzie (2009-01-06 16:15 +0000) wrote:

> >> After "C-c . awk RET" it changes to this:

> >>     "[ \t]*\\(\\(#+\\)[ \t]*\\)?$\\|^\f"
> >                    ^^

> >> Even though I chose to override the style settings "#*" changes to "#+".

> > I don't think this is a bug.  You asked for "awk" style to be set on
> > the buffer, and this is exactly what you got.

> Well, I asked that, but I had also explicitly asked to override style
> settings (through the customize interface). My opinion is that the
> current behavior is so confusing that it's either a bug or simply wrong
> design decision.

It's very confusing.  I get confused, even though I maintain it and I
wrote the bit of the manual that describes it.  I honestly believe the
current functionality is "over-engineered", simply too feature-rich for
the job it does.  However, it's there and (?lots of) people would
complain loudly if any features were abolished.

The style-variables each have a global default, a value in the current
style and a buffer local value in each buffer (which could be either of
the two previous values).  In turn, the style is either an explicitly
selected style or the default style.  Additionally, this all interacts
with the customisation facility, which has no notion of buffer local
variables.  I admit, starting from scratch, nobody would design it this
way.

The other thing, I don't think it matters too much exactly how the style
system works.  What is bad is when it confuses people, or creates
expectations which won't be satisfied.  (See below.)

> I don't mean to complain too loudly, though. See below.

Complaining loudly is OK on this mailing list.  :-)

> > I am guessing that the cause is in the fine CC Mode manual, [...]

> > [...] perhaps it would be better if amended something like this:

> >     When you initialize the buffer, the settings are made in the
> >     following order.  So if you make conflicting settings in several of
> >     these ways, the way that takes precedence is the one that appears
> >     latest in the list(2):

> >         Style
> >         Top-level command or "customization interface"
> >         Hook
> >         File Style

> >     ---------- Footnotes ----------
> >     (2) If you later call `c-set-style' (C-c .), all the style variables
> >     will get set to the style you select.

> > What do you think?

> Well, my real opinion first: User's explicit "override style settings"
> setting should just do that: override style settings no matter what
> style changes the user does while editing the buffer.

OK, this is the text in the M-x customize-variable screen.

> For example, user may want to choose certain custom comment prefix
> settings and, while editing the code, she may want to experiment with
> different style settings or convert the code from one style to another.
> Logically, if she wants to get comment prefix settings from a
> pre-defined style she chooses "Use style settings"
> (c-comment-prefix-regexp). If she wants to use her own comment prefix
> she chooses "Override style settings". I think this is the most
> intuitive behaviour. Currently user can only override style settings
> for the initial style state.

This could be implemented in `c-set-style', [by looking at the global
value of c-comment-prefix-regexp, and using this value rather than the
style's value if the global value isn't 'set-from-style].  Feel free to
ignore the bit in the brackets if it doesn't make much sense!

But this would mean that c-set-style would not do exactly what it says.
With all due respect, I think this would be at least as un-intuitive as
the current mechanism.  Of course, I could create a new option saying
which of these is to be preferred, but that would add even more confusion
into the mix.

> But if you choose to maintain the current behaviour anyway, then I'd say
> that the changes you suggested for the documentation are very good. I'd
> also suggest to change the documentation of c-comment-prefix-regexp
> variable so that it tells how the variable effects in the practice.

Agreed.  This also applies to the other style variables.  What I think is
buggy at the moment is that:
o - `customize-variable' doesn't say "Global default value" anywhere.
o - "Override style settings" is confusing.  Maybe "Set explicit value"
  would be better.

> Anyway, the empty-line bug is the main thing. If that bug in Awk mode
> gets fixed (you already did with that patch) then there shouldn't be
> much need for changing comment prefixes in the first place. So I'm
> really more on the "thank you" side than on "complaining" side. :-)

Cheers!  I've committed the patch to the Emacs repostory (which will
become the future Emacs-23).

But it would still be nice to get `customize-variable' working nicely
with CC Mode.

-- 
Alan Mackenzie (Nuremberg, Germany).

------------------------------------------------------------------------------
Check out the new SourceForge.net Marketplace.
It is the best place to buy or sell services for
just about anything Open Source.
http://p.sf.net/sfu/Xq1LFB