Finding Unused Identifiers

Finding Unused Identifiers

[August Karlstrom]

Hi everyone,

Is there a way to find all unused identifiers in an Elisp file?


Regards,

August

-- 
I am the "ILOVEGNU" signature virus. Just copy me to your
signature.  This email was infected under the terms of the GNU
General Public License.

Re: Finding Unused Identifiers

[Markus Triska]

Hi August,

August Karlstrom wrote:

> 
> Is there a way to find all unused identifiers in an Elisp file?
> 

There's no programmatic way.

All the best,
Markus.

Re: Finding Unused Identifiers

[B. T. Raven]

"Markus Triska" <triska@gmx.at> wrote in message
news:4406fc36$0$11610$3b214f66@tunews.univie.ac.at...
> Hi August,
>
> August Karlstrom wrote:
>
> >
> > Is there a way to find all unused identifiers in an Elisp file?
> >
>
> There's no programmatic way.
>
> All the best,
> Markus.

I probably don't even understand the question but, out of curiosity I ran
the following in *scratch*:

(boundp 't)
t
(boundp t)
t
(boundp 'v31416)
nil


Why can't the file be loaded, tokenized, and then run the list of tokens
against boundp? Or by "unused" do you mean something other than unbound?

Ed

Re: Finding Unused Identifiers

[Markus Triska]

Hi!

B. T. Raven wrote:

 > by "unused" do you mean something other than unbound?

I took it to mean "(at least) one write access (set, let, ...), and no 
read access".

> Why can't the file be loaded

Consider:

(let ((a 0))
   (complicated-function) ; not involving a
   (print a))

If "complicated-function" halts, a is (subsequently) used. If it 
doesn't, a remains unused (the "print" part is never reached). Since 
whether a Lisp function halts is undecidable, there's no algorithm that 
always both terminates and correctly identifies all such identifiers in 
arbitrary programs.

All the best,
Markus.

Re: Finding Unused Identifiers

[August Karlstrom]

B. T. Raven wrote:
> "Markus Triska" <triska@gmx.at> wrote in message
> news:4406fc36$0$11610$3b214f66@tunews.univie.ac.at...
> 
>>Hi August,
>>
>>August Karlstrom wrote:
>>
>>
>>>Is there a way to find all unused identifiers in an Elisp file?
>>>
>>
>>There's no programmatic way.
>>
>>All the best,
>>Markus.
> 
> 
> I probably don't even understand the question 

I mean "forgotten" variables and functions that were never intended to 
be exported. I know that in Elisp all identifiers with file scope are 
exported, but it would be nice to at least have a list of all 
declared-but-never-used-in-the-same-file identifiers. That would give 
you an hint of what can safely be removed.

> but, out of curiosity I ran
> the following in *scratch*:
> 
> (boundp 't)
> t
> (boundp t)
> t
> (boundp 'v31416)
> nil
> 
> 
> Why can't the file be loaded, tokenized, and then run the list of tokens
> against boundp? Or by "unused" do you mean something other than unbound?
> 
> Ed
> 


August

-- 
I am the "ILOVEGNU" signature virus. Just copy me to your
signature.  This email was infected under the terms of the GNU
General Public License.

Re: Finding Unused Identifiers

[Peter Tury]

On Thu, 02 Mar 2006 22:25:10 GMT, August Karlstrom wrote:

> I mean "forgotten" variables and functions that were never intended to 
> be exported. I know that in Elisp all identifiers with file scope are 
> exported, but it would be nice to at least have a list of all 
> declared-but-never-used-in-the-same-file identifiers. That would give 
> you an hint of what can safely be removed.
> 

As the first easiest(?) step it might help if you generate a list of all
words what has only one instance in that file (and that instance is in an
outermost defvar...)? If the list would have a form what is usable in
compilation mode, then one could go throught it easily and check what
should really be removed...

Br,
P

Re: Finding Unused Identifiers

[Pascal Bourguignon]

Peter Tury <tury.peter@gmail.com> writes:

> On Thu, 02 Mar 2006 22:25:10 GMT, August Karlstrom wrote:
>
>> I mean "forgotten" variables and functions that were never intended to 
>> be exported. I know that in Elisp all identifiers with file scope are 
>> exported, but it would be nice to at least have a list of all 
>> declared-but-never-used-in-the-same-file identifiers. That would give 
>> you an hint of what can safely be removed.
>> 
>
> As the first easiest(?) step it might help if you generate a list of all
> words what has only one instance in that file (and that instance is in an
> outermost defvar...)? If the list would have a form what is usable in
> compilation mode, then one could go throught it easily and check what
> should really be removed...

(require 'cl)
(require 'pjb-sources)
;; http://www.informatimago.com/develop/emacs/                           

(let ((defvars '())
      (refered '()))
  (flet ((push-refered (form)
           (cond ((symbolp form)
                  (push form refered))
                 (t (while (consp form)
                      (push-refered (car form))
                      (setf form (cdr form)))))))
    (map-sexps "/home/pjb/src/public/emacs/pjb-sources.el"
               (lambda (form start end)
                  (if (and (consp form) (eq 'defvar (first form)))
                     (progn (push (second form) defvars)
                            (and (third form) (push-refered (third form))))
                     (push-refered form)))
              :deeply nil :atoms t)
   (print defvars)
   (set-difference defvars refered)))

Prints:

(dummy line-num-map *silent* *pjb-sources-initials* *map-sexps-function*
 *map-sexps-atoms* *map-sexps-deeply* *map-sexps-top-level*
 *walk-sexps-end-marker* update-def-names-minimum-lines update-def-names
 *greek-flk* pretty-greek)

and returns:

(dummy)

                   
-- 
__Pascal Bourguignon__                     http://www.informatimago.com/

WARNING: This product warps space and time in its vicinity.

Re: Finding Unused Identifiers

[Markus Triska]

Hi Pascal,

Pascal Bourguignon wrote:

>    (set-difference defvars refered)))

In this example, ab is referred to although the token "ab" appears only 
once:

(defvar ab 0)

(set (intern (concat "a" "b")) 1)


All the best,
Markus.

Re: Finding Unused Identifiers

[Pascal Bourguignon]

Markus Triska <triska@gmx.at> writes:
> Pascal Bourguignon wrote:
>
>>    (set-difference defvars refered)))
>
> In this example, ab is referred to although the token "ab" appears
> only once:
>
> (defvar ab 0)
>
> (set (intern (concat "a" "b")) 1)

Of course, the programmer who edit his sources must know what he does.

-- 
__Pascal Bourguignon__                     http://www.informatimago.com/
Litter box not here.
You must have moved it again.
I'll poop in the sink. 

Re: Finding Unused Identifiers

[Markus Triska]

Hi Pascal,

Pascal Bourguignon wrote:

> 
> Of course, the programmer who edit his sources must know what he does.
> 

Right. Don't trust any program claiming to implement a general solution 
to OP's problem.

All the best,
Markus.

Re: Finding Unused Identifiers

[August Karlstrom]

Peter Tury wrote:
> As the first easiest(?) step it might help if you generate a list of all
> words what has only one instance in that file (and that instance is in an
> outermost defvar...)? If the list would have a form what is usable in
> compilation mode, then one could go throught it easily and check what
> should really be removed...

I wrote the following, which basically does what I want:

(defconst my-elisp-global-decl-re
   (concat "^[ \t]*("
           (regexp-opt '("defconst" "defimage" "defmacro" "defsubst"
                         "defun" "defvar"))
           "[ \t]+\\<\\(\\w+\\)\\>"))


(defun my-elisp-unused-globals ()
   "Return a list of unused global identifiers.  Returns a list of
global identifiers in the current buffer declared with defconst,
defimage, defmacro, defsubst, defun or defvar that are never
accessed in the same buffer.  Requires that no local identifier
uses the same name as a global identifier."
   (let ((pos nil)
         (res nil)
         (id nil))
     (save-excursion
       (beginning-of-buffer)
       (setq pos (re-search-forward my-elisp-global-decl-re nil t))
       (while (not (null pos))
         (save-excursion
           (setq id (match-string-no-properties 1))
           (setq pos (re-search-forward
                      (concat "\\<" id "\\>") nil t))
           (when (null pos)
             (setq pos (re-search-backward
                        (concat "\\<" id "\\>") nil t 2)))
           (when (null pos)
             (setq res (cons id res))))
         (setq pos (re-search-forward
                    my-elisp-global-decl-re nil t))))
     res))


(defun my-elisp-list-unused-globals ()
   "Display the list returned by `my-elisp-unused-globals'. Output
is written to a separate buffer."
   (interactive)
   (let ((unused-globals (my-elisp-unused-globals))
         (buffer nil))
     (setq buffer (get-buffer-create "*Elisp Unused Globals*"))
     (when (= (count-windows) 1) (split-window))
     (other-window 1)
     (switch-to-buffer buffer)
     (erase-buffer)
     (if (null unused-globals)
         (insert "No unused global identifiers found.")
       (insert "Found unused global identifier(s):\n\n")
       (mapc '(lambda (x) (insert x) (newline)) unused-globals)
       (switch-to-buffer buffer))))


August

-- 
I am the "ILOVEGNU" signature virus. Just copy me to your
signature.  This email was infected under the terms of the GNU
General Public License.

Re: Finding Unused Identifiers

[August Karlstrom]

August Karlstrom wrote:
> (defun my-elisp-list-unused-globals ()
>   "Display the list returned by `my-elisp-unused-globals'. Output
> is written to a separate buffer."
>   (interactive)
>   (let ((unused-globals (my-elisp-unused-globals))
>         (buffer nil))
>     (setq buffer (get-buffer-create "*Elisp Unused Globals*"))
>     (when (= (count-windows) 1) (split-window))
>     (other-window 1)
>     (switch-to-buffer buffer)
>     (erase-buffer)
>     (if (null unused-globals)
>         (insert "No unused global identifiers found.")
>       (insert "Found unused global identifier(s):\n\n")
>       (mapc '(lambda (x) (insert x) (newline)) unused-globals)
>       (switch-to-buffer buffer))))

The last line is (of course) incorrect. Should be

(defun my-elisp-list-unused-globals ()
   (interactive)
   (let ((unused-globals (my-elisp-unused-globals))
         (buffer nil))
     (setq buffer (get-buffer-create "*Elisp Unused Globals*"))
     (when (= (count-windows) 1) (split-window))
     (other-window 1)
     (switch-to-buffer buffer)
     (erase-buffer)
     (if (null unused-globals)
         (insert "No unused global identifiers found.")
       (insert "Found unused global identifier(s):\n\n")
       (mapc '(lambda (x) (insert x) (newline)) unused-globals))
     (other-window -1)))


August

-- 
I am the "ILOVEGNU" signature virus. Just copy me to your
signature.  This email was infected under the terms of the GNU
General Public License.

Re: Finding Unused Identifiers

[Markus Triska]

Hi August,

August Karlstrom wrote:

> The last line is (of course) incorrect. Should be

The comment "that are never accessed in the same buffer" is also incorrect:

(defvar falsepositive 0)

(eval `(print ,(intern (concat "false" "positive"))))

==>

Found unused global identifier(s):

falsepositive


Hyphenated identifiers can yield false negatives:

(defvar has-hyphen 0)

(print 'has)

==>

No unused global identifiers found.

All the best,
Markus.

Re: Finding Unused Identifiers

[August Karlstrom]

Markus Triska wrote:
> The comment "that are never accessed in the same buffer" is also incorrect:
> 
> (defvar falsepositive 0)
> 
> (eval `(print ,(intern (concat "false" "positive"))))
> 
> ==>
> 
> Found unused global identifier(s):
> 
> falsepositive
> 
> 
> Hyphenated identifiers can yield false negatives:
> 
> (defvar has-hyphen 0)
> 
> (print 'has)
> 
> ==>
> 
> No unused global identifiers found.

Thanks Markus for your remarks. When I tested my functions I had 
previously changed the syntax class for `-' to "w" (word) and then 
forgot that, so I never noticed the problem. I wonder why the syntax 
class of dash is not "word" by default in LISP buffers.

Here is a corrected version:

(defun my-elisp-unused-globals ()
   "Return a list of unused global identifiers.  Returns a list of
global identifiers in the current buffer declared with defconst,
defimage, defmacro, defsubst, defun or defvar that occur only
once in the same buffer.  Requires that no local identifier uses
the same name as a global identifier."
   (let ((pos nil)
         (res nil)
         (id nil)
         (saved-syntax (char-syntax ?-)))
     (modify-syntax-entry ?- "w")
     (save-excursion
       (beginning-of-buffer)
       (setq pos (re-search-forward my-elisp-global-decl-re nil t))
       (while (not (null pos))
         (save-excursion
           (setq id (match-string-no-properties 1))
           (setq pos (re-search-forward (concat "\\<" id "\\>") nil t))
           (when (null pos)
             (setq pos (re-search-backward (concat "\\<" id "\\>")
                                           nil t 2)))
           (when (null pos)
             (setq res (cons id res))))
         (setq pos (re-search-forward my-elisp-global-decl-re nil t))))
     (modify-syntax-entry ?- (string saved-syntax))
     res))


Regards,

August

-- 
I am the "ILOVEGNU" signature virus. Just copy me to your
signature.  This email was infected under the terms of the GNU
General Public License.

Re: Finding Unused Identifiers

[Kevin Rodgers]

August Karlstrom wrote:
 > Thanks Markus for your remarks. When I tested my functions I had
 > previously changed the syntax class for `-' to "w" (word) and then
 > forgot that, so I never noticed the problem. I wonder why the syntax
 > class of dash is not "word" by default in LISP buffers.

Because "-" is a symbol constituent, not a word constituent:

  - Syntax class: word constituent
      "Word constituents" (designated by `w') are parts of normal
      English words and are typically used in variable and command names
      in programs.  All upper- and lower-case letters, and the digits,
      are typically word constituents.

  - Syntax class: symbol constituent
      "Symbol constituents" (designated by `_') are the extra characters
      that are used in variable and command names along with word
      constituents.  For example, the symbol constituents class is used
      in Lisp mode to indicate that certain characters may be part of
      symbol names even though they are not part of English words.
      These characters are `$&*+-_<>'.  In standard C, the only
      non-word-constituent character that is valid in symbols is
      underscore (`_').

-- 
Kevin Rodgers

Re: Finding Unused Identifiers

[August Karlstrom]

Kevin Rodgers wrote:
> August Karlstrom wrote:
>  > Thanks Markus for your remarks. When I tested my functions I had
>  > previously changed the syntax class for `-' to "w" (word) and then
>  > forgot that, so I never noticed the problem. I wonder why the syntax
>  > class of dash is not "word" by default in LISP buffers.
> 
> Because "-" is a symbol constituent, not a word constituent:
> 
>  - Syntax class: word constituent
>      "Word constituents" (designated by `w') are parts of normal
>      English words and are typically used in variable and command names
>      in programs.  All upper- and lower-case letters, and the digits,
>      are typically word constituents.
> 
>  - Syntax class: symbol constituent
>      "Symbol constituents" (designated by `_') are the extra characters
>      that are used in variable and command names along with word
>      constituents.  For example, the symbol constituents class is used
>      in Lisp mode to indicate that certain characters may be part of
>      symbol names even though they are not part of English words.
>      These characters are `$&*+-_<>'.  In standard C, the only
>      non-word-constituent character that is valid in symbols is
>      underscore (`_').

OK, thanks for pointing that out. So what is the shortest regular 
expression to match an identifier containing characters with symbol 
syntax? Does every such character have to be listed as in e.g.

    \\<[[:word:]_-]+\\>

?


August

-- 
I am the "ILOVEGNU" signature virus. Just copy me to your
signature.  This email was infected under the terms of the GNU
General Public License.