suppressing byte-compiler warnings about undefined functions

suppressing byte-compiler warnings about undefined functions

[Glenn Morris]

Here's an attempt at a method to allow for suppressing of
byte-compiler warnings about undefined functions.

I haven't yet written the function to check that functions are
actually defined in the specified files, but I imagine it would be
straightforward.

*** byte-run.el	26 Jul 2007 05:26:44 -0000	1.22
--- byte-run.el	10 Nov 2007 00:32:42 -0000
***************
*** 103,108 ****
--- 103,118 ----
       (eval-and-compile
         (put ',name 'byte-optimizer 'byte-compile-inline-expand))))
  
+ (defun declare-function (fn file)
+   "Tell the byte-compiler that function FN is defined, in FILE.
+ The FILE argument is not used by the byte-compiler, but by the
+ function `check-declared-functions', which checks that FILE
+ contains a definition for FN.  FILE should be either absolute, or
+ relative to the location of the file containing the declaration, e.g.
+ 
+ \(declare-function 'c-end-of-defun \"progmodes/cc-cmds.el\")"
+   nil)
+ 
  (defun make-obsolete (obsolete-name current-name &optional when)
    "Make the byte-compiler warn that OBSOLETE-NAME is obsolete.
  The warning will say that CURRENT-NAME should be used instead.
*** bytecomp.el	9 Nov 2007 15:57:46 -0000	2.216
--- bytecomp.el	10 Nov 2007 00:46:23 -0000
***************
*** 534,539 ****
--- 534,542 ----
  Used for warnings when a function is not known to be defined or is later
  defined with incorrect args.")
  
+ (defvar byte-compile-declared-functions nil
+   "List of functions constructed by `declare-function' calls when compiling.")
+ 
  (defvar byte-compile-noruntime-functions nil
    "Alist of functions called that may not be defined when the compiled code is run.
  Used for warnings about calling a function that is defined during compilation
***************
*** 1472,1479 ****
        ;; Separate the functions that will not be available at runtime
        ;; from the truly unresolved ones.
        (dolist (f byte-compile-unresolved-functions)
! 	(setq f (car f))
! 	(if (fboundp f) (push f noruntime) (push f unresolved)))
        ;; Complain about the no-run-time functions
        (byte-compile-print-syms
         "the function `%s' might not be defined at runtime."
--- 1475,1482 ----
        ;; Separate the functions that will not be available at runtime
        ;; from the truly unresolved ones.
        (dolist (f byte-compile-unresolved-functions)
!         (unless (memq (setq f (car f)) byte-compile-declared-functions)
!           (if (fboundp f) (push f noruntime) (push f unresolved))))
        ;; Complain about the no-run-time functions
        (byte-compile-print-syms
         "the function `%s' might not be defined at runtime."
***************
*** 1895,1901 ****
  	;; would be useful to delay this warning until all have been
  	;; compiled.  A: Yes!  b-c-u-f might contain dross from a
  	;; previous byte-compile.
! 	(setq byte-compile-unresolved-functions nil)
  
  	;; Compile the forms from the input buffer.
  	(while (progn
--- 1898,1905 ----
  	;; would be useful to delay this warning until all have been
  	;; compiled.  A: Yes!  b-c-u-f might contain dross from a
  	;; previous byte-compile.
! 	(setq byte-compile-unresolved-functions nil
! 	      byte-compile-declared-functions nil)
  
  	;; Compile the forms from the input buffer.
  	(while (progn
***************
*** 2818,2823 ****
--- 2822,2834 ----
  	(body
  	 (list body))))
  
+ ;; Add an element to byte-compile-declared-functions.
+ (put 'declare-function 'byte-hunk-handler 'byte-compile-declare-function)
+ (defun byte-compile-declare-function (form)
+   (push (eval (cadr form)) byte-compile-declared-functions)
+   nil)
+ 
+ \f
  ;; This is the recursive entry point for compiling each subform of an
  ;; expression.
  ;; If for-effect is non-nil, byte-compile-form will output a byte-discard

Re: suppressing byte-compiler warnings about undefined functions

[Stefan Monnier]

> + (defun declare-function (fn file)

Any reason not to include the arglist (optionally)?


        Stefan

Re: suppressing byte-compiler warnings about undefined functions

[Richard Stallman]

    I haven't yet written the function to check that functions are
    actually defined in the specified files, but I imagine it would be
    straightforward.

It is probably nontrivial to arrange to find all the `declare-function'
calls in Emacs and check them.  I doubt it requires deep thought,
but it should to be implemented before we install this.

	    ;; would be useful to delay this warning until all have been
	    ;; compiled.  A: Yes!  b-c-u-f might contain dross from a
	    ;; previous byte-compile.
    ! 	(setq byte-compile-unresolved-functions nil
    ! 	      byte-compile-declared-functions nil)

Why clear byte-compile-declared-functions at the end of compilation?
That would cause strange results if there is an error in compilation.
The reliable way is to do it at the beginning of compilation.
Or let-bind the variable in the right places.

Re: suppressing byte-compiler warnings about undefined functions

[Glenn Morris]

Richard Stallman wrote:

> It is probably nontrivial to arrange to find all the
> `declare-function' calls in Emacs and check them. I doubt it
> requires deep thought, but it should to be implemented before we
> install this.

See below: `check-declared-functions'. I haven't bothered to make it
efficient/elegant (see how long it takes when all the necessary
declare statements are added...). I'm thinking it could go in
admin/admin.el (it uses process-lines from there).

> Why clear byte-compile-declared-functions at the end of compilation?

I'm using the existing variable `byte-compile-function-environment'
now. This also allows for the optional argument checking Stefan asked for.


*** byte-run.el	26 Jul 2007 05:26:44 -0000	1.22
--- byte-run.el	11 Nov 2007 00:35:08 -0000
***************
*** 103,108 ****
--- 103,126 ----
       (eval-and-compile
         (put ',name 'byte-optimizer 'byte-compile-inline-expand))))
  
+ (defun declare-function (fn file &optional arglist)
+   "Tell the byte-compiler that function FN is defined, in FILE.
+ Optional ARGLIST is the argument list used by the function.  The
+ FILE argument is not used by the byte-compiler, but by the
+ function `check-declared-functions', which checks that FILE
+ contains a definition for FN.  FILE should be either absolute, or
+ relative to the location of the file containing the declaration.
+ ARGLIST is used by both the byte-compiler and
+ `check-declared-functions' to check for consistency.
+ 
+ Note that for the purposes of `check-declared-functions', this
+ statement must be the first non-whitespace on a line, and
+ everything up to the end of FILE must be all on the same line.
+ For example:
+ 
+ \(declare-function 'c-end-of-defun \"progmodes/cc-cmds.el\" '(&optional arg))"
+ nil)
+ 
  (defun make-obsolete (obsolete-name current-name &optional when)
    "Make the byte-compiler warn that OBSOLETE-NAME is obsolete.
  The warning will say that CURRENT-NAME should be used instead.
*** bytecomp.el	10 Nov 2007 08:05:15 -0000	2.217
--- bytecomp.el	11 Nov 2007 00:35:23 -0000
***************
*** 1258,1264 ****
  		  (byte-compile-fdefinition (car form) t)))
  	 (sig (if (and def (not (eq def t)))
  		  (byte-compile-arglist-signature
! 		   (if (eq 'lambda (car-safe def))
  		       (nth 1 def)
  		     (if (byte-code-function-p def)
  			 (aref def 0)
--- 1258,1264 ----
  		  (byte-compile-fdefinition (car form) t)))
  	 (sig (if (and def (not (eq def t)))
  		  (byte-compile-arglist-signature
! 		   (if (memq (car-safe def) '(declared lambda))
  		       (nth 1 def)
  		     (if (byte-code-function-p def)
  			 (aref def 0)
***************
*** 2818,2823 ****
--- 2818,2831 ----
  	(body
  	 (list body))))
  
+ (put 'declare-function 'byte-hunk-handler 'byte-compile-declare-function)
+ (defun byte-compile-declare-function (form)
+   (push (cons (eval (nth 1 form))
+               (list 'declared (eval (nth 3 form))))
+         byte-compile-function-environment)
+   nil)
+ 
+ \f
  ;; This is the recursive entry point for compiling each subform of an
  ;; expression.
  ;; If for-effect is non-nil, byte-compile-form will output a byte-discard




;; Adapted from authors.el.
(defmacro checkdec-visit (file &rest body)
  "Execute the forms in BODY while visiting FILE.
Re-uses an existing buffer visiting FILE if there is one.  The
value returned is the value of the last form in BODY."
  (declare (indent 1))
  `(let ((existing-buffer (find-buffer-visiting ,file))
         (enable-local-variables :safe)
         (enable-local-eval nil)
         (buffer (find-file-noselect ,file)))
     (prog1
         (save-current-buffer
           (set-buffer buffer)
           (save-restriction
             (widen)
             (goto-char (point-min))
             ,@body))
       (unless existing-buffer
         (kill-buffer buffer)))))

(defun checkdec-warn (file fn fnfile type)
  "Warn that FILE made a false claim about FN in FNFILE.
TYPE is a string given the nature of the error."
  (display-warning 'checkdec
                   (format "%s said `%s' was defined in %s: %s"
                           (file-name-nondirectory file) fn
                           (file-relative-name fnfile
                                               (file-name-directory file))
                           type)
                   nil "*Check Declarations Warnings*"))

(autoload 'byte-compile-arglist-signature "bytecomp.el")

(defun checkdec-verify (file fn fnfile &optional arglist)
  "Check that FNFILE defines the function FN, as claimed in FILE.
Optionally also check that the arglist matches ARGLIST.
Returns non-nil if the claim was incorrect in some way."
  (unless (file-name-absolute-p fnfile)
    (setq fnfile (expand-file-name fnfile (file-name-directory file))))
  (let (type defarglist)
    (if (file-exists-p fnfile)
        (checkdec-visit fnfile
          (if (re-search-forward
               (format "^(def\\(un\\|subst\\|macro\\)[ \t]+%s\\>" fn) nil t)
              (when arglist
                (skip-chars-forward " \t")
                (if (eolp) (forward-line 1))
                (skip-chars-forward " \t")
                (if (looking-at "(")
                    (setq defarglist (read (current-buffer))))
                (or (equal (byte-compile-arglist-signature defarglist)
                           (byte-compile-arglist-signature arglist))
                    (setq type "arglist mismatch")))
            (setq type "function not found")))
      (setq type "file not found"))
    (if type
        (checkdec-warn file fn fnfile type))))

(defun checkdec-scan (file)
  "Scan FILE for `declare-function' calls and check them.
Returns non-nil if any checks fail."
  (let ((m (format "Checking %s..." file))
        alist anyf e)
     (message "%s" m)
     (checkdec-visit file
      (while (re-search-forward
              "^[ \t]*(declare-function[ \t]+'\\(\\S-+\\)[ \t]+\
\"\\(\\S-+\\)\"" nil t)
        (setq e (list (match-string-no-properties 1)
                      (match-string-no-properties 2)))
        (skip-chars-forward " \t")
        (if (eolp) (forward-line 1))
        (skip-chars-forward " \t'")
        (if (looking-at "(")
            (setq e (append e (list (read (current-buffer))))))
        (setq alist (cons e alist))))
    (dolist (e alist)
      (setq anyf (or anyf
                     (checkdec-verify file (car e) (nth 1 e) (nth 2 e)))))
    (message "%sdone" m)
    anyf))

(defun check-declared-functions (root)
  "Check veracity of all `declare-function' statements under directory ROOT.
Returns non-nil if any false statements are found.  For this to
work correctly, the statements must adhere to the format
described in the documentation of `declare-function'."
  (interactive "DEmacs lisp directory: ")
  (setq root (expand-file-name root))
  (unless (file-exists-p (expand-file-name "emacs-lisp/bytecomp.el" root))
    (error "Not the root lisp directory of Emacs: %s" root))
  (let ((m "Checking `declare-function' statements...")
        anyf)
    (message "%s" m)
    (dolist (file (process-lines "find" root "-name" "*.el"
                                 "-exec" "grep" "-l"
                                 "^(declare-function" "{}" ";"))
      (setq anyf (or anyf (checkdec-scan file))))
    (message "%s%s" m (if anyf "problems found" "OK"))
    anyf))

Re: suppressing byte-compiler warnings about undefined functions

[Glenn Morris]

Glenn Morris wrote:

> See below: `check-declared-functions'.

Try and ignore the silly mistakes related to `anyf'... :(

Re: suppressing byte-compiler warnings about undefined functions

[Alan Mackenzie]

On Fri, Nov 09, 2007 at 07:55:08PM -0500, Glenn Morris wrote:
> 
> Here's an attempt at a method to allow for suppressing of
> byte-compiler warnings about undefined functions.
 
> I haven't yet written the function to check that functions are
> actually defined in the specified files, but I imagine it would be
> straightforward.
 
As a matter of interest, there has been this sort of functionality in
cc-bytecomp.el (written by Martin Stjernholm) for some time.  In
particular, `cc-bytecomp-defun' and `cc-bytecomp-defvar', whose prime
use is suppressing compiler warnings.

It might be sensible to reuse this.

-- 
Alan Mackenzie (Ittersbach, Germany).

Re: suppressing byte-compiler warnings about undefined functions

[Dan Nicolaescu]

Alan Mackenzie <acm@muc.de> writes:

  > On Fri, Nov 09, 2007 at 07:55:08PM -0500, Glenn Morris wrote:
  > > 
  > > Here's an attempt at a method to allow for suppressing of
  > > byte-compiler warnings about undefined functions.
  >  
  > > I haven't yet written the function to check that functions are
  > > actually defined in the specified files, but I imagine it would be
  > > straightforward.
  >  
  > As a matter of interest, there has been this sort of functionality in
  > cc-bytecomp.el (written by Martin Stjernholm) for some time.  In
  > particular, `cc-bytecomp-defun' and `cc-bytecomp-defvar', whose prime
  > use is suppressing compiler warnings.

What is the advantage of (cc-bytecomp-defvar VARIABLE) over plain
(defvar VARIABLE)? 

  > It might be sensible to reuse this.

Would that be possible without editing all elisp files in emacs?

Re: suppressing byte-compiler warnings about undefined functions

[Alan Mackenzie]

Hi, Dan!

On Sun, Nov 11, 2007 at 07:51:24AM -0800, Dan Nicolaescu wrote:
> Alan Mackenzie <acm@muc.de> writes:

>   > On Fri, Nov 09, 2007 at 07:55:08PM -0500, Glenn Morris wrote:

>   > > Here's an attempt at a method to allow for suppressing of
>   > > byte-compiler warnings about undefined functions.

>   > > I haven't yet written the function to check that functions are
>   > > actually defined in the specified files, but I imagine it would be
>   > > straightforward.

>   > As a matter of interest, there has been this sort of functionality
>   > in cc-bytecomp.el (written by Martin Stjernholm) for some time.  In
>   > particular, `cc-bytecomp-defun' and `cc-bytecomp-defvar', whose
>   > prime use is suppressing compiler warnings.

> What is the advantage of (cc-bytecomp-defvar VARIABLE) over plain
> (defvar VARIABLE)? 

You can have the variable "defined" solely for the duration of the
byte-compilation.  This is useful for `xemacs-foo' and `gnu-emacs-bar'
if your file.el is portable.  It's useful for core Emacs
variables/functions defined in a file which isn't currently loaded.

>   > It might be sensible to reuse this.

> Would that be possible without editing all elisp files in emacs?

Yes, absolutely!  But it'll need some files to be edited.  ;-)

-- 
Alan Mackenzie (Ittersbach, Germany).

Re: suppressing byte-compiler warnings about undefined functions

[Dan Nicolaescu]

Alan Mackenzie <acm@muc.de> writes:

  > Hi, Dan!
  > 
  > On Sun, Nov 11, 2007 at 07:51:24AM -0800, Dan Nicolaescu wrote:
  > > Alan Mackenzie <acm@muc.de> writes:
  > 
  > >   > On Fri, Nov 09, 2007 at 07:55:08PM -0500, Glenn Morris wrote:
  > 
  > >   > > Here's an attempt at a method to allow for suppressing of
  > >   > > byte-compiler warnings about undefined functions.
  > 
  > >   > > I haven't yet written the function to check that functions are
  > >   > > actually defined in the specified files, but I imagine it would be
  > >   > > straightforward.
  > 
  > >   > As a matter of interest, there has been this sort of functionality
  > >   > in cc-bytecomp.el (written by Martin Stjernholm) for some time.  In
  > >   > particular, `cc-bytecomp-defun' and `cc-bytecomp-defvar', whose
  > >   > prime use is suppressing compiler warnings.
  > 
  > > What is the advantage of (cc-bytecomp-defvar VARIABLE) over plain
  > > (defvar VARIABLE)? 
  > 
  > You can have the variable "defined" solely for the duration of the
  > byte-compilation.  This is useful for `xemacs-foo' and `gnu-emacs-bar'
  > if your file.el is portable.  It's useful for core Emacs
  > variables/functions defined in a file which isn't currently loaded.

But that is not different from plain defvar, it just seems to add
a layer of obfuscation...

  > >   > It might be sensible to reuse this.
  > 
  > > Would that be possible without editing all elisp files in emacs?
  > 
  > Yes, absolutely!  But it'll need some files to be edited.  ;-)

:-) Joke aside, please see the long discussion that preceded Glenn's
design, the thread was called "byte compiler warnings when bootstrapping".

Re: suppressing byte-compiler warnings about undefined functions

[Dan Nicolaescu]

Dan Nicolaescu <dann@ics.uci.edu> writes:

  > Alan Mackenzie <acm@muc.de> writes:
  > 
  >   > Hi, Dan!
  >   > 
  >   > On Sun, Nov 11, 2007 at 07:51:24AM -0800, Dan Nicolaescu wrote:
  >   > > Alan Mackenzie <acm@muc.de> writes:
  >   > 
  >   > >   > On Fri, Nov 09, 2007 at 07:55:08PM -0500, Glenn Morris wrote:
  >   > 
  >   > >   > > Here's an attempt at a method to allow for suppressing of
  >   > >   > > byte-compiler warnings about undefined functions.
  >   > 
  >   > >   > > I haven't yet written the function to check that functions are
  >   > >   > > actually defined in the specified files, but I imagine it would be
  >   > >   > > straightforward.
  >   > 
  >   > >   > As a matter of interest, there has been this sort of functionality
  >   > >   > in cc-bytecomp.el (written by Martin Stjernholm) for some time.  In
  >   > >   > particular, `cc-bytecomp-defun' and `cc-bytecomp-defvar', whose
  >   > >   > prime use is suppressing compiler warnings.
  >   > 
  >   > > What is the advantage of (cc-bytecomp-defvar VARIABLE) over plain
  >   > > (defvar VARIABLE)? 
  >   > 
  >   > You can have the variable "defined" solely for the duration of the
  >   > byte-compilation.  This is useful for `xemacs-foo' and `gnu-emacs-bar'
  >   > if your file.el is portable.  It's useful for core Emacs
  >   > variables/functions defined in a file which isn't currently loaded.
  > 
  > But that is not different from plain defvar, it just seems to add
  > a layer of obfuscation...

In fact most cc-bytecomp-defvars are not useful. I removed all of them
and recompiled. Only filladapt-*, c-syntactic-* and
c-preprocessor-face-name got warnings.

Re: suppressing byte-compiler warnings about undefined functions

[Richard Stallman]

    + (defun declare-function (fn file &optional arglist)

Why not &rest?

    + Note that for the purposes of `check-declared-functions', this
    + statement must be the first non-whitespace on a line, and
    + everything up to the end of FILE must be all on the same line.
    + For example:
    + 
    + \(declare-function 'c-end-of-defun \"progmodes/cc-cmds.el\" '(&optional arg))"

Breaking the line after the file name
would make the example more helpful.

    + Note that for the purposes of `check-declared-functions', this
    + statement must be the first non-whitespace on a line, and
    + everything up to the end of FILE must be all on the same line.
    + For example:
    + 
    + \(declare-function 'c-end-of-defun \"progmodes/cc-cmds.el\" '(&optional arg))"
    + nil)
    + 
      (defun make-obsolete (obsolete-name current-name &optional when)
	"Make the byte-compiler warn that OBSOLETE-NAME is obsolete.
      The warning will say that CURRENT-NAME should be used instead.
    *** bytecomp.el	10 Nov 2007 08:05:15 -0000	2.217
    --- bytecomp.el	11 Nov 2007 00:35:23 -0000
    ***************
    *** 1258,1264 ****
		      (byte-compile-fdefinition (car form) t)))
	     (sig (if (and def (not (eq def t)))
		      (byte-compile-arglist-signature
    ! 		   (if (eq 'lambda (car-safe def))
			   (nth 1 def)
			 (if (byte-code-function-p def)
			     (aref def 0)
    --- 1258,1264 ----
		      (byte-compile-fdefinition (car form) t)))
	     (sig (if (and def (not (eq def t)))
		      (byte-compile-arglist-signature
    ! 		   (if (memq (car-safe def) '(declared lambda))
			   (nth 1 def)
			 (if (byte-code-function-p def)
			     (aref def 0)
    ***************
    *** 2818,2823 ****
    --- 2818,2831 ----
	    (body
	     (list body))))

    + (put 'declare-function 'byte-hunk-handler 'byte-compile-declare-function)
    + (defun byte-compile-declare-function (form)
    +   (push (cons (eval (nth 1 form))
    +               (list 'declared (eval (nth 3 form))))
    +         byte-compile-function-environment)
    +   nil)
    + 
    + \f
      ;; This is the recursive entry point for compiling each subform of an
      ;; expression.
      ;; If for-effect is non-nil, byte-compile-form will output a byte-discard




    ;; Adapted from authors.el.
    (defmacro checkdec-visit (file &rest body)
      "Execute the forms in BODY while visiting FILE.
    Re-uses an existing buffer visiting FILE if there is one.  The
    value returned is the value of the last form in BODY."
      (declare (indent 1))
      `(let ((existing-buffer (find-buffer-visiting ,file))
	     (enable-local-variables :safe)
	     (enable-local-eval nil)
	     (buffer (find-file-noselect ,file)))

This would run a lot faster if it just reads the file
into a temporary buffer and avoids visiting.


checkdoc-scan could be far far more efficient.  It should scan all
files for `declare-function' calls, sort them by the file that is
supposed to define them, then load each file just once to verify.

Re: suppressing byte-compiler warnings about undefined functions

[Richard Stallman]

      > But that is not different from plain defvar, it just seems to add
      > a layer of obfuscation...

    In fact most cc-bytecomp-defvars are not useful. I removed all of them
    and recompiled. Only filladapt-*, c-syntactic-* and
    c-preprocessor-face-name got warnings.

The point is that (defvar foo) does the same job.
It turns off warnings only in the file it appears in.

Re: suppressing byte-compiler warnings about undefined functions

[Glenn Morris]

Richard Stallman wrote:

>     + (defun declare-function (fn file &optional arglist)
>
> Why not &rest?

Because I want to specify the arglist as a single quoted list.

> Breaking the line after the file name would make the example more
> helpful.

OK.

> This would run a lot faster if it just reads the file into a
> temporary buffer and avoids visiting.

Yes. I based the first implementation on authors.el, which uses
visiting for some reason (maybe it needs to process file local
variables to get the right coding system for author names, or
something).

> checkdoc-scan could be far far more efficient. It should scan all
> files for `declare-function' calls, sort them by the file that is
> supposed to define them, then load each file just once to verify.

OK, here's a version that works like that.

This could be used for things other than Emacs, so I no longer think
it should go in admin.el. Where could it be installed?

I would also like to move process-lines out of admin.el and make it
more generally available (in subr.el?). It is used by this code, and
also by authors.el.

I will also install a Makefile rule such that one can use `make
checkdecs' to verify the Emacs lisp/ directory.


(defconst checkdec-warning-buffer "*Check Declarations Warnings*"
  "Name of buffer used to display any checkdec warnings.")

(defun checkdec-scan (file)
  "Scan FILE for `declare-function' calls.
Return a list with elements of the form (FNFILE FN ARGLIST), where
ARGLIST may be absent.  This claims that FNFILE defines FN, with ARGLIST."
  (let ((m (format "Scanning %s..." file))
        alist fnfile fn)
    (message "%s" m)
    (with-temp-buffer
      (insert-file-contents file)
      (while (re-search-forward
              "^[ \t]*(declare-function[ \t]+'\\(\\S-+\\)[ \t]+\
\"\\(\\S-+\\)\"" nil t)
        (setq fn (match-string 1)
              fnfile (match-string 2))
        (or (file-name-absolute-p fnfile)
            (setq fnfile (expand-file-name fnfile (file-name-directory file))))
        (setq alist (cons
                     (list fnfile fn
                           (progn
                             (skip-chars-forward " \t\n'")
                             ;; Use `t' to distinguish no arglist
                             ;; specified from an empty one.
                             (if (looking-at "(")
                                 (read (current-buffer))
                               t)))
                     alist))))
    (message "%sdone" m)
    alist))

(autoload 'byte-compile-arglist-signature "bytecomp")

(defun checkdec-verify (fnfile fnlist)
  "Check that FNFILE contains function definitions matching FNLIST.
Each element of FNLIST has the form (FILE FN ARGLIST), where
ARGLIST is optional.  This means FILE claimed FN was defined in
FNFILE with the specified ARGLIST.  Returns nil if all claims are
found to be true, otherwise a list of errors with elements of the form
\(FILE FN TYPE), where TYPE is a string giving details of the error."
  (let ((m (format "Checking %s..." fnfile))
        re fn sig siglist arglist type errlist)
    (message "%s" m)
    (if (file-exists-p fnfile)
        (with-temp-buffer
          (insert-file-contents fnfile)
          (setq re (format "^(defun[ \t]+%s\\>"
                           (regexp-opt (mapcar (lambda (e) (cadr e)) fnlist)
                                       t)))
          (while (re-search-forward re nil t)
            (skip-chars-forward " \t\n")
            (setq fn (match-string 1)
                  sig (if (looking-at "(") (byte-compile-arglist-signature
                                            (read (current-buffer))))
                  ;; alist of functions and arglist signatures.
                  siglist (cons (cons fn sig) siglist)))))
    (dolist (e fnlist)
      (setq arglist (nth 2 e)
            type
            (if re                     ; re non-nil means found a file
                (if (setq sig (assoc (cadr e) siglist))
                    ;; Recall we use t to mean no arglist specified,
                    ;; to distinguish from an empty arglist.
                    (unless (eq arglist t)
                      (unless (equal (byte-compile-arglist-signature arglist)
                                     (cdr sig))
                        "arglist mismatch"))
                  "function not found")
              "file not found"))
      (when type
        (setq errlist (cons (list (car e) (cadr e) type) errlist))))
    (message "%s%s" m (if errlist "problems found" "OK"))
    errlist))

(defun checkdec-sort (alist)
  "Sort a list with elements FILE (FNFILE ...).
Returned list has elements FNFILE (FILE ...)."
  (let (file fnfile rest sort a)
    (dolist (e alist)
      (setq file (car e))
      (dolist (f (cdr e))
        (setq fnfile (car f)
              rest (cdr f))
        (if (setq a (assoc fnfile sort))
            (setcdr a (append (cdr a) (list (cons file rest))))
          (setq sort (cons (list fnfile (cons file rest)) sort)))))
    sort))

(defun checkdec-warn (file fn fnfile type)
  "Warn that FILE made a false claim about FN in FNFILE.
TYPE is a string giving the nature of the error.  Warning is displayed in
`checkdec-warning-buffer'."
  (display-warning 'checkdec
                   (format "%s said `%s' was defined in %s: %s"
                           (file-name-nondirectory file) fn
                           (file-name-nondirectory fnfile)
                           type)
                   nil checkdec-warning-buffer))

;;;###autoload
(defun checkdec-file (file)
  "Check veracity of all `declare-function' statements in FILE.
See `checkdec-directory' for more information."
  (interactive "fFile to check: ")
  (or (file-exists-p file)
      (error "file `%s' not found" file))
  (let ((m (format "Checking %s..." file))
        err errlist)
    (message "%s" m)
    (dolist (e (checkdec-sort (list (cons file (checkdec-scan file)))))
      (if (setq err (checkdec-verify (car e) (cdr e)))
          (setq errlist (cons (cons (car e) err) errlist))))
    (if (get-buffer checkdec-warning-buffer)
        (kill-buffer checkdec-warning-buffer))
    (dolist (e (checkdec-sort errlist))
      (dolist (f (cdr e))
        (checkdec-warn (car e) (cadr f) (car f) (nth 2 f))))
    (message "%s%s" m (if errlist "problems found" "OK"))
    errlist))

;;;###autoload
(defun checkdec-directory (root)
  "Check veracity of all `declare-function' statements under directory ROOT.
Returns non-nil if any false statements are found.  For this to
work correctly, the statements must adhere to the format
described in the documentation of `declare-function'."
  (interactive "DEroot lisp directory: ")
  (setq root (expand-file-name root))
  (let ((m "Checking `declare-function' statements...")
        (m2 "Finding files with declarations...")
        alist err errlist files)
    (message "%s" m)
    (message "%s" m2)
    (setq files (process-lines "find" root "-name" "*.el"
                                 "-exec" "grep" "-l"
                                 "^[ 	]*(declare-function" "{}" ";"))
    (message "%s%d found" m2 (length files))
    (when files
      (dolist (file files)
        (setq alist (cons (cons file (checkdec-scan file)) alist)))
      ;; Sort so that things are ordered by the files supposed to
      ;; contain the defuns.
      (dolist (e (checkdec-sort alist))
        (if (setq err (checkdec-verify (car e) (cdr e)))
            (setq errlist (cons (cons (car e) err) errlist))))
      (if (get-buffer checkdec-warning-buffer)
          (kill-buffer checkdec-warning-buffer))
      ;; Sort back again so that errors are ordered by the files
      ;; containing the declare-function statements.
      (dolist (e (checkdec-sort errlist))
        (dolist (f (cdr e))
          (checkdec-warn (car e) (cadr f) (car f) (nth 2 f))))
      (message "%s%s" m (if errlist "problems found" "OK"))
      errlist)))

Re: suppressing byte-compiler warnings about undefined functions

[martin rudalics]

 >>checkdoc-scan could be far far more efficient. It should scan all
         ^

`checkdec' will be likely confused more often with `checkdoc'.  I think
a different term should be used.  Also, wouldn't it be reasonable to add
`declare-variable' as well?  I'm not arguing for consistency here.  But
we could get rid of comments talking about "byte-compiler warnings" or
"pacify byte-compiler".  And the intention of some `(defvar variable)'s
without such comments would become more clear.

Re: suppressing byte-compiler warnings about undefined functions

[Richard Stallman]

    >     + (defun declare-function (fn file &optional arglist)
    >
    > Why not &rest?

    Because I want to specify the arglist as a single quoted list.

Yes, indeed, we don't want to have to quote each arg.
Therefore, for convenient use, I suggest making this
a macro with a &rest argument.

    This could be used for things other than Emacs, so I no longer think
    it should go in admin.el.

That's a bad place.  This code should be included in the Emacs
release.  It should go in lisp/emacs-lisp.

Re: suppressing byte-compiler warnings about undefined functions

[Stefan Monnier]

>     Because I want to specify the arglist as a single quoted list.

> Yes, indeed, we don't want to have to quote each arg.
> Therefore, for convenient use, I suggest making this
> a macro with a &rest argument.

I agree it should be a macro (with an empty body), but I think that
having the arglist as a single arg is better because it makes it
syntactically idential to the arglist in the function definition.


        Stefan

Re: suppressing byte-compiler warnings about undefined functions

[Glenn Morris]

Stefan Monnier wrote:

> I agree it should be a macro (with an empty body), but I think that
> having the arglist as a single arg is better because it makes it
> syntactically idential to the arglist in the function definition.

I agree with all that and will make it a macro.

Re: suppressing byte-compiler warnings about undefined functions

[Glenn Morris]

martin rudalics wrote:

> `checkdec' will be likely confused more often with `checkdoc'.  I think
> a different term should be used.

I suppose it is a bit similar. Any suggestions anyone?

> Also, wouldn't it be reasonable to add `declare-variable' as well?
> I'm not arguing for consistency here. But we could get rid of
> comments talking about "byte-compiler warnings" or "pacify
> byte-compiler". And the intention of some `(defvar variable)'s
> without such comments would become more clear.

Personally I don't have a problem with `(defvar foo)'.

Re: suppressing byte-compiler warnings about undefined functions

[Stefan Monnier]

>> `checkdec' will be likely confused more often with `checkdoc'.  I think
>> a different term should be used.

> I suppose it is a bit similar. Any suggestions anyone?

crosscheck ?

>> Also, wouldn't it be reasonable to add `declare-variable' as well?
>> I'm not arguing for consistency here. But we could get rid of
>> comments talking about "byte-compiler warnings" or "pacify
>> byte-compiler". And the intention of some `(defvar variable)'s
>> without such comments would become more clear.

> Personally I don't have a problem with `(defvar foo)'.

Same here.


        Stefan

Re: suppressing byte-compiler warnings about undefined functions

[Richard Stallman]

    I agree it should be a macro (with an empty body), but I think that
    having the arglist as a single arg is better because it makes it
    syntactically idential to the arglist in the function definition.

To me, the fact that there is no body makes that extra level of parens
seem like unnecessary clutter.  But I won't put up a fight about the
question.

Re: suppressing byte-compiler warnings about undefined functions

[Richard Stallman]

    > `checkdec' will be likely confused more often with `checkdoc'.  I think
    > a different term should be used.

    I suppose it is a bit similar. Any suggestions anyone?

How about `check-declarations'?

People won't call this by hand very often.
The name need not be short.

Re: suppressing byte-compiler warnings about undefined functions

[Glenn Morris]

Richard Stallman wrote:

> How about `check-declarations'?

Installed as `check-declare'.

New make target: make check-declare

Re: suppressing byte-compiler warnings about undefined functions

[Richard Stallman]

    > How about `check-declarations'?

    Installed as `check-declare'.

That is good.

byte-compiler warnings about undefined functions can now be silenced

[Glenn Morris]

Dan Nicolaescu wrote:

> Could you please post a message to emacs-devel about how to use this
> feature?
> 
> An example with the warning that is currently produced by the byte
> compiler and how to use this to get rid of it.

As requested...


When bootstrapping, there are lots of messages about functions that
are "not known to be defined". The compiler is technically correct, but
the code is usually such that when it actually runs, the function will
be defined.

For example, byte-compiling fortran.el used to warn:

  In end of data:
  fortran.el:2152:1:Warning: the function `gud-find-c-expr' is not known
    to be defined.

But gud-find-c-expr was only used in the function that fortran mode
uses for the local value of `gud-find-expr-function'. So this would
only ever be called from gud, so the warning can safely be suppressed.
It's nice to do this, so that real warnings are more visible.

All you need to do is add a `declare-function' statement before the
first use of the function in question:

  (declare-function gud-find-c-expr "gud.el" nil)

This says that gud-find-c-expr is defined in "gud.el" (the `.el' can
be omitted). The file path is either absolute, or relative to the one
with the declare-function statement (e.g. "../files.el").

The 3rd argument is optional, and specifies the argument list of
gud-find-c-expr. In this case, it takes no arguments (`nil' is
different from not specifying a value). In other cases, this might be
something like (file &optional overwrite). You don't have to specify
the argument list, but if you do the byte-compiler will check that the
calls match the declaration.


The functions `check-declare-file' and `check-declare-directory' will
check that all the declare-function statements in a file or directory
are true (i.e. that the functions are defined in the specified files,
and have the same argument lists, if specified). `make check-declare'
will check all of leim/ and lisp/.

> Also, IMO it would be good to add declare-function to the 22.2
> branch, (even as an empty stub if it's considered too risky), this
> should help with compatibility between the 2 branches.

I don't have much of an opinion. It could easily be added as a no-op
(it's almost a no-op now...).

Re: byte-compiler warnings about undefined functions can now be silenced

[martin rudalics]

>>Could you please post a message to emacs-devel about how to use this
>>feature?
>>
>>An example with the warning that is currently produced by the byte
>>compiler and how to use this to get rid of it.
> 
> 
> As requested...

Please put it in a comment of check-declare.el.

Re: byte-compiler warnings about undefined functions can now be silenced

[Richard Stallman]

    > Also, IMO it would be good to add declare-function to the 22.2
    > branch, (even as an empty stub if it's considered too risky), this
    > should help with compatibility between the 2 branches.

    I don't have much of an opinion. It could easily be added as a no-op
    (it's almost a no-op now...).

I think that is the best approach.

Re: byte-compiler warnings about undefined functions can now be silenced

[Jay Belanger]

> Dan Nicolaescu wrote:
...
> When bootstrapping, there are lots of messages about functions that
> are "not known to be defined". The compiler is technically correct, but
> the code is usually such that when it actually runs, the function will
> be defined.
...
> All you need to do is add a `declare-function' statement before the
> first use of the function in question:
>
>   (declare-function gud-find-c-expr "gud.el" nil)
>
> This says that gud-find-c-expr is defined in "gud.el" (the `.el' can
> be omitted). The file path is either absolute, or relative to the one
> with the declare-function statement (e.g. "../files.el").
...
> You don't have to specify the argument list, but if you do the
> byte-compiler will check that the calls match the declaration.

Is there any reason the filename isn't optional, too?  `check-declare-file'
wouldn't be able to check anything in that case, of course, but it could
still keep the byte-compiler quiet.

Jay

Re: byte-compiler warnings about undefined functions can now be silenced

[Glenn Morris]

"Jay Belanger" wrote:

> Is there any reason the filename isn't optional, too?  `check-declare-file'
> wouldn't be able to check anything in that case, of course, but it could
> still keep the byte-compiler quiet.

It's my recollection that rms specifically wanted it to work this way,
but I haven't bothered to check the mailing list archives...

Re: byte-compiler warnings about undefined functions can now be silenced

[Glenn Morris]

Richard Stallman wrote:

>     I don't have much of an opinion. It could easily be added as a no-op
>     (it's almost a no-op now...).
>
> I think that is the best approach.

Added in 22 as an alias for ignore.

Re: byte-compiler warnings about undefined functions can now be silenced

[Richard Stallman]

    Is there any reason the filename isn't optional, too?

I don't recall thinking specifically about it before, but a good
reason is so that people won't omit the file name out of laziness.

Re: byte-compiler warnings about undefined functions can now be silenced

[Glenn Morris]

martin rudalics wrote:

> Please put it in a comment of check-declare.el.

Per rms request, now in lispref.