Emacs Lisp Linter fail to identify function defined in a "use-package" block

Emacs Lisp Linter fail to identify function defined in a "use-package" block

[Kiong-Gē Liāu]

[-- Attachment #1: Type: text/plain, Size: 712 bytes --]

Hi, 

  I found that the built-in Emacs lisp linter seems not able to
identify functions defined within a (even the same) use-package :config
block as the attached screenshot indicates: the "elisp-face-temp"
function is defined right above the "elisp-face" function, however, the
linter says that the "`elisp-face-temp' is not known to be defined".
Are there some further configurations required in order to make the
linter identifies functions defined in nested structure or we need to
change the linter's implementation code to make  the linter to identify
functions those are not defined at top-level (i.e., defun block within
a use-package or with-eval-after-loan block)

Thanks, 
Kiong-Gē. 

[-- Attachment #2: Screenshot from 2023-10-10 09-37-53.png --]
[-- Type: image/png, Size: 284855 bytes --]

Re: Emacs Lisp Linter fail to identify function defined in a "use-package" block

[Emanuel Berg]

Kiong-Gē Liāu wrote:

> [...] or we need to change the linter's implementation code
> to make the linter to identify functions those are not
> defined at top-level (i.e., defun block within a use-package
> or with-eval-after-loan block)

Or a defun in a lexical let-closure, which currently the
byte-compiler don't see, so one has to do, e.g.

  (declare-function set-x-clipboard nil)

See if you can do the same thing and if that shuts up the
linter as well.

This is obviously not optimal, the byte-compiler, and the
linter as well, all tho I have no experience from it, should
be able to see them as they are in use in the global space
just like any top-level defun.

-- 
underground experts united
https://dataswamp.org/~incal

Re: Emacs Lisp Linter fail to identify function defined in a "use-package" block

[Emanuel Berg]

Kiong-Gē Liāu wrote:

> I found that the built-in Emacs lisp linter [...]

If we are talking `elint-current-buffer', I just tried it on
a couple of my Elisp files. Most of them has warnings of the
type

  Reference to unbound symbol

or

  Setting previously unbound symbol

and indeed, even running it on elint.el itself produces such
warnings - but these warnings are not reported by the
byte-compiler and also do not cause problems in practice.

I don't know what one is supposed to do to with them,
`require' even more files than the byte-compiler demands?

It also reports errors like this for functions with &rest
arguments, apparently the linter does think they
are mandatory,

  Wrong number of args: (cl-remove 32 (string-to-list abc)),
  (ITEM SEQ [KEYWORD &rest VALUE])

Elint was written in 1997, it is likely the situation was
quite different back then, maybe that is the explanation.

But I think the byte compiler should be the de facto linter,
instead of improving elint.el (and `checkdoc' for that matter)
whatever good stuff are in those should be incorporated into
the byte compiler to have all the tidying up at one place.

Now we also get warnings from the native-compiler - where does
it all end? In one and only one place - would be optimal for
me, at least.

But cred to Emacs hackers be it 1997 or 2023!

-- 
underground experts united
https://dataswamp.org/~incal

Re: Emacs Lisp Linter fail to identify function defined in a "use-package" block

[Emanuel Berg]

I sent a mail to

  Author: Peter Liljenberg <petli@lysator.liu.se>

as stated in elint.el but the mail bounced.

-- 
underground experts united
https://dataswamp.org/~incal

Re: Emacs Lisp Linter fail to identify function defined in a "use-package" block

[Kiong-Gē Liāu]

[-- Attachment #1: Type: text/plain, Size: 1069 bytes --]

Hi Emanuel,

I did some further tests and found that the "function *** is not known
to be defined" error only happen once for either a undefined function
or a function defined in a nested structure, as seen in the attached
screenshot.

1. The kkl0-fun, which is defined at top-level, linter shows no error
   at all about it
2. The kkl1-fun is defined as long as the *kk0l* global variable 
   is assigned as t
3. The first linter error appears when the kkl1-fun is called first
   time
4. At the second call-site of kkl1-fun, the linter "undefined-unction"
   error disappears 
5. the kkl2-fun is totally undefined, and linter identifies the
   undefined function error at its first call-site
6. at the second call-site of kkl2-fun, the undefined-function linter
   error disappears as well

So, it seems that elint starts to ignore undefined function starting
second call-site regardless that function is defined or not. we need
some further investigation on elint.el's elint-update-env function and
related functions.

Thanks,
Kiong-Gē.

[-- Attachment #2: Screenshot from 2023-10-11 08-56-48.png --]
[-- Type: image/png, Size: 49918 bytes --]

Re: Emacs Lisp Linter fail to identify function defined in a "use-package" block

[Kiong-Gē Liāu]

[-- Attachment #1: Type: text/plain, Size: 442 bytes --]

Hi,

I just realized that it's caused by flyCheck rather than elint. Sorry for
some misleading information.


I also noticed that elint is unable to identify variables/functions defined
in subr.el, startup.el, or any other el files that has no explicit
"provides" statement at their end. We may need to add some supplemental
information to elint to avoid false-alarms about undefined functions or
variables.

Thanks,
Kiong-Gē.

[-- Attachment #2: Type: text/html, Size: 775 bytes --]

Re: Emacs Lisp Linter fail to identify function defined in a "use-package" block

[Emanuel Berg]

Kiong-Gē Liāu wrote:

> I also noticed that elint is unable to identify
> variables/functions defined in subr.el, startup.el, or any
> other el files that has no explicit "provides" statement at
> their end.

Well, here the discussion can go both ways! Because in a way
it makes sense that every file `provide' its stuff, and every
other file that wants to use it `require' it.

Update 1, elint is correctly seeing defuns in lexical
let-closures, and there is no need to use `declare-function'
for it do that as was incorrectly theorized the day before.
This is interesting because the byte-compiler does not see
those so there declare-function is needed.

Update 2, elint cannot see variables in `pcase-let' when the
variables are defined using the `(,a ,b ,c) syntax - warning
BTW, if that looks like a list with a bunch of items
being evaluated, that isn't the case, here, it is a syntax
used for special purposes by pcase-let.

pcase-let was introduced in Emacs 28.1, probably elint has not
been brought up to date on it.

-- 
underground experts united
https://dataswamp.org/~incal

Re: Emacs Lisp Linter fail to identify function defined in a "use-package" block

[Emanuel Berg]

> Well, here the discussion can go both ways! Because in a way
> it makes sense that every file `provide' its stuff, and
> every other file that wants to use it `require' it.

Yes, this is a huge problem if you want to tidy up one's Elisp
using elint. You get so many warnings this or that variable
isn't known to be defined, you check where it is defined but
that source file isn't `provide'd or in some cases (e.g.
`float-pi') it is incorrectly provided (i.e. it is defined in
float-sup.el but that file provides its services as
"lisp-float-type").

Note that this isn't elint's fault, on the contrary, the fault
is with those files. But what happens is the interesting
situation that one cannot use elint to tidy up one's Elisp,
because the Elisp of vanilla Emacs isn't tidy enough. Not good!

Yeah, all files should `require' everything they need and
provide itself in the end. This situation in itself isn't
optimal - well, as we have just seen, it is error prone -
since it can and should be automated, there is no need to rely
on humans to do this work, however now it isn't and that means
one must get it right manually.

> Update 1, elint is correctly seeing defuns in lexical
> let-closures, and there is no need to use `declare-function'
> for it do that as was incorrectly theorized the day before.
> This is interesting because the byte-compiler does not see
> those so there declare-function is needed.

Unfortunately, this isn't exactly true either. Both elint and
the byte-compiler are silent if a defun in a lexical
let-closure is just defined. However if and where it is
_used_, elint cannot see it, and `define-function' doesn't
help. define-function does shut up the byte-compiler in this
situation tho.

-- 
underground experts united
https://dataswamp.org/~incal

Re: Emacs Lisp Linter fail to identify function defined in a "use-package" block

[Emanuel Berg]

There is also some fail cases I don't understand, for example

  Reference to unbound symbol: dired-directory
  Reference to unbound symbol: dired-mode-map

But they are both in dired.el which is provided correctly and
requiring it doesn't help.

-- 
underground experts united
https://dataswamp.org/~incal

Re: Emacs Lisp Linter fail to identify function defined in a "use-package" block

[Emanuel Berg]

> Update 2, elint cannot see variables in `pcase-let' when the
> variables are defined using the `(,a ,b ,c) syntax - warning
> BTW, if that looks like a list with a bunch of items being
> evaluated, that isn't the case, here, it is a syntax used
> for special purposes by pcase-let.

Same with `pcase', here elint reports the incorrect warning

  Empty varlist in let

but it is a correct, pcase-specific syntax.

-- 
underground experts united
https://dataswamp.org/~incal

Re: Emacs Lisp Linter fail to identify function defined in a "use-package" block

[Emanuel Berg]

> dired [...]

The reason why it doesn't work for dired.el even tho it has
a `provide' last, stating the correct dired, is that it isn't
_quite_ last, I think.

Here are dired.el lines 5115-5119 using Emacs 30.0 [1] 

  (provide 'dired)

  (run-hooks 'dired-load-hook)		; for your customizations

  ;;; dired.el ends here

This is good enough for the byte-compiler, but not good enough
for elint.

[1] GNU Emacs 30.0.50 (build 1, x86_64-pc-linux-gnu, cairo
    version 1.16.0) of 2023-10-10 [commit
    239db5d5162f6f9a4a6735e176c8d306f18e50e7]

-- 
underground experts united
https://dataswamp.org/~incal

Re: Emacs Lisp Linter fail to identify function defined in a "use-package" block

[Emanuel Berg]

I have now used elint on all my Elisp, totalling
196 files and 10 907 sloc.

The problems with elint, using `elint-current-buffer', are:

  - &rest arguments
  - `pcase'
  - `pcase-let'
  - lexical `let'-closures

The problems with vanilla Emacs source files, that are brought
to our attention thanks to elint, are:

  - they do not `provide' (e.g. startup.el)
  - they provide incorrectly (float-sup.el)
  - they don't use provide good enough (dired.el)

Of these problems, the byte-compiler, which is also a de facto
Elisp linter, only has the problem with lexical let-closures.
That particular problem can be remedied with
`declare-function' - but shouldn't have to be, and that method
doesn't work for elint.

The problem with vanilla Emacs sources files are not reported
by the byte-compiler.

-- 
underground experts united
https://dataswamp.org/~incal

Re: Emacs Lisp Linter fail to identify function defined in a "use-package" block

[Emanuel Berg]

> The problems with vanilla Emacs source files, that are
> brought to our attention thanks to elint, are:
>
>   - they do not `provide' (e.g. startup.el) [...]

A work-around is to simply `defvar' the missing variables.

I don't know if the linter has succeeded in improving the
code, but I added to .emacs this,

  (defvar auto-save-list-file-prefix)
  (defvar backup-inhibited)
  (defvar inhibit-default-init)
  (defvar inhibit-startup-screen)
  (defvar initial-scratch-message)
  (defvar make-backup-files)
  (defvar user-emacs-directory)

and Elint doesn't mention them.

-- 
underground experts united
https://dataswamp.org/~incal

Re: Emacs Lisp Linter fail to identify function defined in a "use-package" block

[Emanuel Berg]

I have now tried another Elisp linter which I found on MELPA,
namely elisp-lint version 20220419.252 - you can get it here

  https://github.com/gonewest818/elisp-lint/

In the description, they say you can use it like this

  $(EMACS) -Q --batch -l elisp-lint.el -f elisp-lint-files-batch *.el

however that command does not include the full path of the
program, nor the paths of its dependencies, so it isn't quiet
that easy. But after putting together this mammoth command,
I got it to work

#! /bin/zsh
#
# this file:
#   https://dataswamp.org/~incal/conf/.zsh/emacs

elint () {
    local fs=($@)
    local elpa=~/.emacs.d/elpa
    local elint=${elpa}/elisp-lint-20220419.252/elisp-lint.el
    emacs -Q      \
          --batch \
          --eval "(push \"$elpa/compat-29.1.4.2\" load-path)"            \
          --eval "(push \"$elpa/dash-20230714.723\" load-path)"          \
          --eval "(push \"$elpa/package-lint-20231103.1810\" load-path)" \
          -l $elint                 \
          -f elisp-lint-files-batch \
          --no-byte-compile         \
          --no-check-declare        \
          --no-checkdoc             \
          --no-fill-column          \
          --no-indent               \
          --no-indent-character     \
          --no-package-lint         \
                                    $fs
}

--no-byte-compile - that would involve having to include even
more paths, and also, I already do byte-compile from another
command (in a Makefile) so I thought I'd get those warnings
from there anyway;

--no-check-declare - this remarks on the use of
`declare-function' to get away with byte-compiler warnings
from functions defined in lexical let-closures. I agree that
use shouldn't be necessary, but it is, so to not get the
warning from something put there to not get a warning,
I disabled it;

--no-checkdoc and --no-package-lint - those don't auto-detect
what files are packages and which are not, that is a pity;

and finally these three, which all have to do with
indentation, I disabled them since I don't consider those any
areas of meaningful improvements, really,

  --no-fill-column          
  --no-indent               
  --no-indent-character     

After doing all that, I'm not sure how many test remains!
But my Elisp came out on the other end with no warnings, so
I suppose it is good enough then.

Conclusion: This took some fiddling to get working but I think
it is a powerful program and I like that you can fine-tune it
to your needs in a very granular way.

PS. I CC this to the maintainers, let's hear what they have
    to say.

-- 
underground experts united
https://dataswamp.org/~incal

Re: Emacs Lisp Linter fail to identify function defined in a "use-package" block

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

We don't try to maintain MELPA packages -- they are not part of Emacs.

I suppose this lint program is useful; it must report some potential
problems.  But instead of writing a lint program for Emacs, maybe we
can do this job better by integrating that functionality into the
Emacs Lisp compiler.

That is what I did for C.

The original `lint' program was part of Unix in the 1980s.  It
reported possible wrrors that the Unix C compiler did not try to
detect.

I thought that the need for a separate `lint' program was a design
flaw, so instead of writing a free `lint' for GNU, I made GCC report
those possible errors.  Thus, the GNU system did not need a `lint'
program.  You only needed to run one program over your C source, that
being GCC, and it would do both jobs.  That proved to be a big
improvement.

Whatever the potential problems are that the Emacs Lisp `lint'
reports, can we make the byte compiler report them?

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Emacs Lisp Linter fail to identify function defined in a "use-package" block

[Emanuel Berg]

Richard Stallman wrote:

> I thought that the need for a separate `lint' program was
> a design flaw, so instead of writing a free `lint' for GNU,
> I made GCC report those possible errors. Thus, the GNU
> system did not need a `lint' program. You only needed to run
> one program over your C source, that being GCC, and it would
> do both jobs. That proved to be a big improvement.

I agree it would be beneficial to have it all in one and only
one place.

In vanilla Emacs, warnings and suggestions are or can be
outputted from:

- checkdoc.el
- elint.el
- the byte compiler
- the native compiler

In MELPA there are:

- elisp-lint [1]
- package-lint [2]

elisp-lint has the advantage that it is a meta-linter of sort,
as it also outputs warnings from the byte compiler, from
checkdoc, and package-lint.

[1] https://github.com/gonewest818/elisp-lint/
[2] https://github.com/purcell/package-lint

-- 
underground experts united
https://dataswamp.org/~incal