bug#23313: 25.0.92; CC Mode not recognizing function declarations that returns pointer to custom type

bug#23313: 25.0.92; CC Mode not recognizing function declarations that returns pointer to custom type

[Mohammed Sadik]

c-mode doesn't recognize (or at least colorize as per current rules)
function declarations that returns a pointer (or pointer to pointer) to
custom data types:

Eg., If my c file contains:

     GtkWidget *
     my_window_new (GtkApplication *app);

Its not recognized as a function declaration, while this works:

     GtkWidget
     my_window_new (GtkApplication *app);

Package: CC Mode 5.33

In GNU Emacs 25.0.92.1 (x86_64-unknown-linux-gnu, GTK+ Version 3.18.9)
 of 2016-04-13 built on fedora.localdomain
Windowing system distributor 'Fedora Project', version 11.0.11800000
Configured features:
XPM JPEG TIFF GIF PNG SOUND DBUS GSETTINGS NOTIFY ACL LIBSELINUX GNUTLS
LIBXML2 FREETYPE XFT ZLIB TOOLKIT_SCROLL_BARS GTK3 X11

Important settings:
  value of $LC_MONETARY: en_IN.UTF-8
  value of $LC_NUMERIC: en_IN.UTF-8
  value of $LC_TIME: en_IN.UTF-8
  value of $LANG: en_US.UTF-8
  value of $XMODIFIERS: @im=ibus
  locale-coding-system: utf-8-unix

Major mode: C/l

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t
  abbrev-mode: t

Recent messages:
Mark set
Formatting bug report buffer.......
Please enter your report.  Type C-c C-c to send, C-c C-k to abort.
You can run the command ‘c-submit-bug-report’ with C-c C-b
Please enter your report.  Type C-c C-c to send, C-c C-k to abort.
Auto-saving...done
Mark set
Quit
Making completion list... [3 times]
delete-backward-char: Text is read-only

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug jka-compr info sendmail message dired
format-spec rfc822 mml mml-sec epg epg-config gnus-util mm-decode
mm-bodies mm-encode mail-parse rfc2231 rfc2047 rfc2045 ietf-drums
mm-util help-fns mail-prsvr mailabbrev mail-utils gmm-utils mailheader
reporter cl-extra help-mode cc-mode cc-fonts easymenu cc-guess cc-menus
cc-cmds cc-styles cc-align cc-engine cc-vars cc-defs cl-loaddefs pcase
cl-lib time-date mule-util tooltip eldoc electric uniquify ediff-hook
vc-hooks lisp-float-type mwheel x-win term/common-win x-dnd tool-bar dnd
fontset image regexp-opt fringe tabulated-list newcomment elisp-mode
lisp-mode prog-mode register page menu-bar rfn-eshadow timer select
scroll-bar mouse jit-lock font-lock syntax facemenu font-core frame
cl-generic cham georgian utf-8-lang misc-lang vietnamese tibetan thai
tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek romanian
slovak czech european ethiopic indian cyrillic chinese charscript
case-table epa-hook jka-cmpr-hook help simple abbrev minibuffer
cl-preloaded nadvice loaddefs button faces cus-face macroexp files
text-properties overlay sha1 md5 base64 format env code-pages mule
custom widget hashtable-print-readable backquote dbusbind inotify
dynamic-setting system-font-setting font-render-setting move-toolbar gtk
x-toolkit x multi-tty make-network-process emacs)

Memory information:
((conses 16 115494 15743)
 (symbols 48 22007 0)
 (miscs 40 97 312)
 (strings 32 21591 3488)
 (string-bytes 1 722478)
 (vectors 16 15416)
 (vector-slots 8 462596 12072)
 (floats 8 174 325)
 (intervals 56 547 7)
 (buffers 976 15)
 (heap 1024 49428 2421))

bug#23313: 25.0.92; CC Mode not recognizing function declarations that returns pointer to custom type

[Alan Mackenzie]

Hello, Mohammed.

In article <mailman.657.1461074349.7477.bug-gnu-emacs@gnu.org> you wrote:
> c-mode doesn't recognize (or at least colorize as per current rules)
> function declarations that returns a pointer (or pointer to pointer) to
> custom data types:

> Eg., If my c file contains:

>      GtkWidget *
>      my_window_new (GtkApplication *app);

> Its not recognized as a function declaration, while this works:

>      GtkWidget
>      my_window_new (GtkApplication *app);

Yes.

I think the reason for this is that the first expression is ambiguous.
It could be the function declaration you want it to be, or it could be
the arithmetic expression "GtkWidget times the result of calling
my_window_new with argument GtkApplication times app".  We need to be
careful that expressions like that don't get fontified as declarations by
mistake.

My current idea for resolving this ambiguity is to examine the token
before GtkWidget; if this is a ";" or a "}" or a "{", then we have a
declaration, if it's something else, we have an arithmetic expression.
Or something like that.

Give me a little time, and I'll see what I can come up with.

> Package: CC Mode 5.33

> In GNU Emacs 25.0.92.1 (x86_64-unknown-linux-gnu, GTK+ Version 3.18.9)
>  of 2016-04-13 built on fedora.localdomain
> Windowing system distributor 'Fedora Project', version 11.0.11800000
> Configured features:
> XPM JPEG TIFF GIF PNG SOUND DBUS GSETTINGS NOTIFY ACL LIBSELINUX GNUTLS
> LIBXML2 FREETYPE XFT ZLIB TOOLKIT_SCROLL_BARS GTK3 X11

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#23313: 25.0.92; CC Mode not recognizing function declarations that returns pointer to custom type

[Alan Mackenzie]

Hello again, Mohammed.

On Sat, Apr 23, 2016 at 11:48:30AM -0000, Alan Mackenzie wrote:
> In article <mailman.657.1461074349.7477.bug-gnu-emacs@gnu.org> you wrote:
> > c-mode doesn't recognize (or at least colorize as per current rules)
> > function declarations that returns a pointer (or pointer to pointer) to
> > custom data types:

> > Eg., If my c file contains:

> >      GtkWidget *
> >      my_window_new (GtkApplication *app);

> > Its not recognized as a function declaration, while this works:

> >      GtkWidget
> >      my_window_new (GtkApplication *app);

> Yes.

> I think the reason for this is that the first expression is ambiguous.
> It could be the function declaration you want it to be, or it could be
> the arithmetic expression "GtkWidget times the result of calling
> my_window_new with argument GtkApplication times app".  We need to be
> careful that expressions like that don't get fontified as declarations by
> mistake.

> My current idea for resolving this ambiguity is to examine the token
> before GtkWidget; if this is a ";" or a "}" or a "{", then we have a
> declaration, if it's something else, we have an arithmetic expression.
> Or something like that.

> Give me a little time, and I'll see what I can come up with.

OK, I've written a trial patch along these lines.  Could you try it out,
please, and let me know whether it works OK, or still has some faults.



diff --git a/lisp/progmodes/cc-engine.el b/lisp/progmodes/cc-engine.el
index 62bc236..d72b345 100644
--- a/lisp/progmodes/cc-engine.el
+++ b/lisp/progmodes/cc-engine.el
@@ -7087,6 +7087,9 @@ c-forward-decl-or-cast-1
 	backup-if-not-cast
 	;; For casts, the return position.
 	cast-end
+	;; Set when the symbol before `preceding-token-end' is known to
+	;; terminate the previous construct, or when we're at point-min.
+	at-decl-start
 	;; Have we got a new-style C++11 "auto"?
 	new-style-auto
 	;; Save `c-record-type-identifiers' and
@@ -7096,6 +7099,15 @@ c-forward-decl-or-cast-1
 	(save-rec-type-ids c-record-type-identifiers)
 	(save-rec-ref-ids c-record-ref-identifiers))
 
+    (save-excursion
+      (goto-char preceding-token-end)
+      (setq at-decl-start
+	    (or (bobp)
+		(let ((tok-end (point)))
+		  (c-backward-token-2)
+		  (member (buffer-substring-no-properties (point) tok-end)
+			  c-pre-start-tokens)))))
+
     (while (c-forward-annotation)
       (c-forward-syntactic-ws))
 
@@ -7696,12 +7708,15 @@ c-forward-decl-or-cast-1
 			  at-type
 			  (or at-decl-end (looking-at "=[^=]"))
 			  (not context)
-			  (not got-suffix))
-		 ;; Got something like "foo * bar;".  Since we're not inside an
-		 ;; arglist it would be a meaningless expression because the
-		 ;; result isn't used.  We therefore choose to recognize it as
-		 ;; a declaration.  Do not allow a suffix since it could then
-		 ;; be a function call.
+			  (or (not got-suffix)
+			      at-decl-start))
+		 ;; Got something like "foo * bar;".  Since we're not inside
+		 ;; an arglist it would be a meaningless expression because
+		 ;; the result isn't used.  We therefore choose to recognize
+		 ;; it as a declaration.  We only allow a suffix (which makes
+		 ;; the construct look like a function call) when
+		 ;; `at-decl-start' provides additional evidence that we do
+		 ;; have a declaration.
 		 (throw 'at-decl-or-cast t))
 
 	       ;; CASE 17
diff --git a/lisp/progmodes/cc-langs.el b/lisp/progmodes/cc-langs.el
index dd1bccf..c548394 100644
--- a/lisp/progmodes/cc-langs.el
+++ b/lisp/progmodes/cc-langs.el
@@ -1325,6 +1325,14 @@ 'c-opt-op-identitier-prefix
 (c-lang-defconst c-haskell-op-re
   t (c-make-keywords-re nil (c-lang-const c-haskell-op)))
 (c-lang-defvar c-haskell-op-re (c-lang-const c-haskell-op-re))
+
+(c-lang-defconst c-pre-start-tokens
+  "List of operators following which an apparent declaration \(e.g.
+\"t1 *fn (t2 *b);\") is most likely to be an actual declaration
+\(as opposed to an arithmetic expression)."
+  t '(";" "{" "}"))
+(c-lang-defvar c-pre-start-tokens (c-lang-const c-pre-start-tokens))
+
 \f
 ;;; Syntactic whitespace.
 


> > Package: CC Mode 5.33

> > In GNU Emacs 25.0.92.1 (x86_64-unknown-linux-gnu, GTK+ Version 3.18.9)
> >  of 2016-04-13 built on fedora.localdomain
> > Windowing system distributor 'Fedora Project', version 11.0.11800000
> > Configured features:
> > XPM JPEG TIFF GIF PNG SOUND DBUS GSETTINGS NOTIFY ACL LIBSELINUX GNUTLS
> > LIBXML2 FREETYPE XFT ZLIB TOOLKIT_SCROLL_BARS GTK3 X11

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#23313: 25.0.92; CC Mode not recognizing function declarations that returns pointer to custom type

[Mohammed Sadik]

On 4/23/16, Alan Mackenzie <acm@muc.de> wrote:

> OK, I've written a trial patch along these lines.  Could you try it out,
> please, and let me know whether it works OK, or still has some faults.
>
 <snip>

The patch was not applying to my Emacs 25.0.93 source. patched by hand.
 And I tried your patch. It was working right, with some minor glitches:

*  The right face is applied only after the statement is made complete
with a ';'

GtkWidget
my_app_window (|)

The above code is interpreted as a function (without ';') but the one
below doesn't:

GtkWidget *
my_app_window (|)

There might be issues guessing the right way. That is, if the * is for
representing a
pointer or mathematical multiplication, but the following is also not
interpreted right
(when without ';'):

GtkWidget **
my_app_window (|)

As ** is clearly a pointer to pointer,  cc-mode shall be able to guess
very early.

I'm not an expert, though the following interpretation might be good:
1. GtkWidget |
   'GtkWidget' may be an identifier, data type, etc.

2. GtkWidget *|
   'GtkWidget' _may_ be a data type (as '*' _may_ not be present to represent
   multiplication as its is not allowed as lvalue).

3. GtkWidget **|
   'GtkWidget' is surely a data type. (not implemented now in c-mode)

4. GtkWidget * my_app_window|
   (2) and my_app_window may be an identifier.

5. GtkWidget * my_app_window (|
   'GtkWidget' may be a data type and 'my_app_window' may be a function.

6.  GtkWidget * my_app_window (int|
    This is not a function call, as 'int' is a data type. So probably
    a function definition/declaration).

Regards
    Mohammed Sadik

bug#23313: 25.0.92; CC Mode not recognizing function declarations that returns pointer to custom type

[Alan Mackenzie]

Hello, Mohammed.

On Sat, Apr 23, 2016 at 10:17:13PM +0530, Mohammed Sadik wrote:
> On 4/23/16, Alan Mackenzie <acm@muc.de> wrote:

> > OK, I've written a trial patch along these lines.  Could you try it out,
> > please, and let me know whether it works OK, or still has some faults.

>  <snip>

> The patch was not applying to my Emacs 25.0.93 source. patched by hand.
>  And I tried your patch. It was working right, with some minor glitches:

Sorry about the bad patch.  I made it to apply to the emacs-25 branch.
Obviously, I made a mistake somewhere.

> *  The right face is applied only after the statement is made complete
> with a ';'

> GtkWidget
> my_app_window (|)

> The above code is interpreted as a function (without ';') but the one
> below doesn't:

> GtkWidget *
> my_app_window (|)

> There might be issues guessing the right way. That is, if the * is for
> representing a
> pointer or mathematical multiplication, ....

That's precisely the issue.  I've tried removing the condition for the
semicolon.  Unfortunately, CC Mode then misfontifies both of the
following (from a CC Mode test file) as declarations:

    t3  *id  (NULL) + 1;
    t3  (id) (NULL) + 1;	

> .... but the following is also not interpreted right (when without
> ';'):

> GtkWidget **
> my_app_window (|)

> As ** is clearly a pointer to pointer,  cc-mode shall be able to guess
> very early.

It's not actually that clear.  The second star could be accessing the
pointer returned by calling my_app_window.  ;-)

> I'm not an expert, though the following interpretation might be good:
> 1. GtkWidget |
>    'GtkWidget' may be an identifier, data type, etc.

> 2. GtkWidget *|
>    'GtkWidget' _may_ be a data type (as '*' _may_ not be present to represent
>    multiplication as its is not allowed as lvalue).

> 3. GtkWidget **|
>    'GtkWidget' is surely a data type. (not implemented now in c-mode)

See above.

> 4. GtkWidget * my_app_window|
>    (2) and my_app_window may be an identifier.

> 5. GtkWidget * my_app_window (|
>    'GtkWidget' may be a data type and 'my_app_window' may be a function.

> 6.  GtkWidget * my_app_window (int|
>     This is not a function call, as 'int' is a data type. So probably
>     a function definition/declaration).

I'd agree with you on number 6.  Trouble is, most types used are user
defined, and not easy to distinguish from variable names.

I've thought long and hard about this needing a semicolon to fontify an
"ambiguous" declaration.  In the end, I've decided that this is a lesser
evil than leaving valid (if unusual and not very useful) function
invocations wrongly fontified as declarations.  There is also something
positive in reminding the user to type that final semicolon, even if
it's not consistent.  So, sorry about this.

So I think this bug is now fixed.  Assuming I don't hear from you about
any more problems, I'll commit the patch to the master branch, and mark
the bug report as fixed.

Thanks for all the trouble you've taken over it.

> Regards
>     Mohammed Sadik

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#23313: 25.0.92; CC Mode not recognizing function declarations that returns pointer to custom type

[Mohammed Sadik]

On 4/25/16, Alan Mackenzie <acm@muc.de> wrote:
> Hello, Mohammed.
>

<snip>

>
>> GtkWidget **
>> my_app_window (|)
>
>> As ** is clearly a pointer to pointer,  cc-mode shall be able to guess
>> very early.
>
> It's not actually that clear.  The second star could be accessing the
> pointer returned by calling my_app_window.  ;-)
>

That was a nice catch. ;-)

> So I think this bug is now fixed.  Assuming I don't hear from you about
> any more problems, I'll commit the patch to the master branch, and mark
> the bug report as fixed.

I hope this patch is commited to emacs-25 branch too.


> --
> Alan Mackenzie (Nuremberg, Germany).
>

bug#23313: 25.0.92; CC Mode not recognizing function declarations that returns pointer to custom type

[Alan Mackenzie]

Hello again, Mohammed.

On Mon, Apr 25, 2016 at 05:10:32PM +0530, Mohammed Sadik wrote:
> On 4/25/16, Alan Mackenzie <acm@muc.de> wrote:
> <snip>


> >> GtkWidget **
> >> my_app_window (|)

> >> As ** is clearly a pointer to pointer,  cc-mode shall be able to guess
> >> very early.

> > It's not actually that clear.  The second star could be accessing the
> > pointer returned by calling my_app_window.  ;-)


> That was a nice catch. ;-)

:-)

> > So I think this bug is now fixed.  Assuming I don't hear from you about
> > any more problems, I'll commit the patch to the master branch, and mark
> > the bug report as fixed.

> I hope this patch is commited to emacs-25 branch too.

Sadly, no.  The Emacs-25 pretest is at a very late stage (John Wiegley,
the maintainer is hoping to release in July, I think), and only fixes
for particularly nasty/urgent/high impact bugs are being accepted now.
In all honesty, this isn't such a bug - people have been living with it
for a very long time (over 10 years, possibly 20), and it doesn't impede
the use of CC Mode all that much, annoying though it is.  I know that
John would reject my request if I asked to merge it into the emacs-25
branch.

So, sorry about that too.  What I suggest you do is to keep the patch
(even though it doesn't apply cleanly), and when you upgrade to Emacs 25
in the (northern hemisphere) summer, apply the patch and rebuild.  At
least that way your own copy will run with the bug fixed.

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#23313: 25.0.92; CC Mode not recognizing function declarations that returns pointer to custom type

[Mohammed Sadik]

On 4/25/16, Alan Mackenzie <acm@muc.de> wrote:

>> I hope this patch is commited to emacs-25 branch too.
>
> Sadly, no.  The Emacs-25 pretest is at a very late stage (John Wiegley,
> the maintainer is hoping to release in July, I think), and only fixes
> for particularly nasty/urgent/high impact bugs are being accepted now.
> In all honesty, this isn't such a bug - people have been living with it
> for a very long time (over 10 years, possibly 20), and it doesn't impede
> the use of CC Mode all that much, annoying though it is.  I know that
> John would reject my request if I asked to merge it into the emacs-25
> branch.
>
> So, sorry about that too.  What I suggest you do is to keep the patch
> (even though it doesn't apply cleanly), and when you upgrade to Emacs 25
> in the (northern hemisphere) summer, apply the patch and rebuild.  At
> least that way your own copy will run with the bug fixed.
>

Its OK. I understand how important stability is, the very same reason why I love
Debian GNU/Linux (+ its a community centered free software).
I think this bug can be closed.

Hope you won't forget your patch in
http://debbugs.gnu.org/cgi/bugreport.cgi?bug=23308

Thank you.

> --
> Alan Mackenzie (Nuremberg, Germany).
>

bug#23313: 25.0.92; CC Mode not recognizing function declarations that returns pointer to custom type

[Alan Mackenzie]

Bug fixed in the master branch.

-- 
Alan Mackenzie (Nuremberg, Germany).