Supporting newer Emacs features in older Emacs versions without warnings?

Supporting newer Emacs features in older Emacs versions without warnings?

[Adam Porter]

Hi,

I recently added a shortdoc group to taxy.el for Emacs 28.  It works
fine in Emacs 28, but in older Emacs versions, it produces warnings at
compile time, because the shortdoc forms appear to be function calls
with incorrect arguments.  For example:

  ;;;; Documentation group

  ;; Available in Emacs 28.  NOTE: In earlier Emacs versions,
  ;; byte-compiling this section will produce warnings due to the
  ;; shortdoc forms that appear to be function calls.

  (eval-when-compile
    (require 'shortdoc nil t))

  (when (require 'shortdoc nil t)
    (with-no-warnings
      ;; TODO: Remove `with-no-warnings' when requiring Emacs 28+.
      (define-short-documentation-group taxy
        (taxy-flatten
         :eval (taxy-flatten
                (make-taxy
                 :items '(a b c)
                 :taxys (list (make-taxy
                               :items '(d e f)))))))))

This (minus the use of `with-no-warnings') produces warnings like:

  taxy.el:368:6: Warning: taxy-flatten called with 2 arguments, but
    accepts only 1

As well as:

  In end of data:
  taxy.el:367:4: Warning: the function
    ‘define-short-documentation-group’ is not known to be defined.

As you can see, I added `with-no-warnings' to the code, which should
hide these warnings.  But that will also obscure warnings in Emacs 28 if
I make any mistakes in that code, so it leaves me with an either/or
situation: either have false warnings in Emacs <28, or no legitimate
warnings in Emacs 28+.

Also, it's awkward to have to `require' shortdoc twice, first in
`eval-when-compile' for Emacs 28, and then in a `when' to avoid
evaluating the shortdoc definer in older versions.

Is there a cleaner way to do this?  Or could we have some kind of form
to aid the use of optional features in different versions of Emacs?
Maybe something like:

  (compile-when (version< "27.2" emacs-version)
    (define-short-documentation-group ...))

--
Thanks,
Adam

Re: Supporting newer Emacs features in older Emacs versions without warnings?

[Basil L. Contovounesios]

Adam Porter [2021-09-22 20:48 -0500] wrote:

> Is there a cleaner way to do this?

Here's what I do, FWIW:

(with-eval-after-load 'shortdoc
  (declare-function shortdoc-add-function "shortdoc" (group section elem))
  (mapc
   (pcase-lambda (`(,section . ,elems))
     (dolist (elem elems)
       (setq elem `(,(car elem) :no-manual t ,@(cdr elem)))
       (shortdoc-add-function 'foo section elem)))
   '(("Frobnicating"
      (foo-frobnicate
       :no-eval (foo-frobnicate)
       :eg-result-string "...")
      ...)
     ("Bifurcating"
      (foo-bifurcate
       :no-eval (foo-bifurcate)
       :eg-result "...")
      ...)
     ...)))

-- 
Basil

Re: Supporting newer Emacs features in older Emacs versions without warnings?

[Adam Porter]

"Basil L. Contovounesios" <contovob@tcd.ie> writes:

> Adam Porter [2021-09-22 20:48 -0500] wrote:
>
>> Is there a cleaner way to do this?
>
> Here's what I do, FWIW:
>
> (with-eval-after-load 'shortdoc
>   (declare-function shortdoc-add-function "shortdoc" (group section elem))
>   (mapc
>    (pcase-lambda (`(,section . ,elems))
>      (dolist (elem elems)
>        (setq elem `(,(car elem) :no-manual t ,@(cdr elem)))
>        (shortdoc-add-function 'foo section elem)))
>    '(("Frobnicating"
>       (foo-frobnicate
>        :no-eval (foo-frobnicate)
>        :eg-result-string "...")
>       ...)
>      ("Bifurcating"
>       (foo-bifurcate
>        :no-eval (foo-bifurcate)
>        :eg-result "...")
>       ...)
>      ...)))

Hi Basil,

Thanks, that does seem to produce fewer warnings in older Emacs
versions.  (Of course, package-lint complains about using
with-eval-after-load outside of user configuration, but that's not
Emacs's fault.)

Re: Supporting newer Emacs features in older Emacs versions without warnings?

[Stefan Monnier]

Can you guys come up with an idea of how to change shortdoc.el so that
your package can do its job more easily and cleanly?
[ Ideally without having to use eval-after-load nor having to require
shortdoc.  ]


        Stefan

Re: Supporting newer Emacs features in older Emacs versions without warnings?

[Adam Porter]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> Can you guys come up with an idea of how to change shortdoc.el so that
> your package can do its job more easily and cleanly?
> [ Ideally without having to use eval-after-load nor having to require
> shortdoc.  ]

Well, I guess we could do something like:

  (ignore-errors
    (declare-function shortdoc-add-function ...)
    (shortdoc-add-function ...))

But that would still produce compilation warnings on pre-shortdoc
Emacsen.  And what if shortdoc isn't loaded yet?  (Is it always loaded
by default, before packages are initialized?)

ISTM that the general problem is how to support newer Emacs features on
older Emacs versions.  If we had some kind of optional-compilation form,
I guess it would help with that (e.g. we can already use (when...) in a
top-level form, but that doesn't prevent compilation of the code, and
`cl-eval-when' only has three choices, not user-defined ones).

So some kind of `compile-when' or `eval-and-compile-when', in which the
user could specify a test form, might be helpful (or maybe that idea's
too crazy, I don't know).

It might also help with supporting optional packages, e.g. in some
packages of mine that support Helm, I have to break out just a few forms
into a separate package to avoid warnings and errors (even a small
mistake can cause errors at init time, which greatly confuse users).
That's not a big deal, but every extra package is more work for
maintainers, and for users (who have to discover that it even exists
before they can use it).

Re: Supporting newer Emacs features in older Emacs versions without warnings?

[Basil L. Contovounesios]

Stefan Monnier [2021-09-24 09:14 -0400] wrote:

> Can you guys come up with an idea of how to change shortdoc.el so that
> your package can do its job more easily and cleanly?
> [ Ideally without having to use eval-after-load nor having to require
> shortdoc.  ]

Apart from eval-after-load, what other mechanisms do we have that can
register information before a package is loaded?  There's obarray and
there are hooks, is there something else?

The entrypoint command shortdoc-display-group could, every time it is
invoked, scan obarray for certain symbol properties, or run a hook, to
accumulate auxiliary shortdoc forms, in addition to those that were
already permanently stored in shortdoc--groups.

Thoughts, preferences?  Is there another, better way?

(For now I'm just wondering about the registration mechanism;
the API is a separate story.)

Thanks,

-- 
Basil

Re: Supporting newer Emacs features in older Emacs versions without warnings?

[Stefan Monnier]

Basil L. Contovounesios [2021-09-26 17:49:33] wrote:
> Stefan Monnier [2021-09-24 09:14 -0400] wrote:
>> Can you guys come up with an idea of how to change shortdoc.el so that
>> your package can do its job more easily and cleanly?
>> [ Ideally without having to use eval-after-load nor having to require
>> shortdoc.  ]
>
> Apart from eval-after-load, what other mechanisms do we have that can
> register information before a package is loaded?  There's obarray and
> there are hooks, is there something else?

For buffer-local info, the standard way is for the amjor mode to do

    (setq-local <foo> <bar>)

Example of <foo> include `font-lock-defaults` (back when font-lock was
not enabled by default), `outline-regexp`, `imenu-generic-expression`, ...

Here I think we want something that's not buffer-local, so we'd need
something else.  Not sure what's the best option.  Maybe

    (add-hook 'shortdoc-foo-functions #'...)

?

> The entrypoint command shortdoc-display-group could, every time it is
> invoked, scan obarray for certain symbol properties, or run a hook, to
> accumulate auxiliary shortdoc forms, in addition to those that were
> already permanently stored in shortdoc--groups.

That's also an option, yes (scanning `obarray` is fairly quick).


        Stefan

Re: Supporting newer Emacs features in older Emacs versions without warnings?

[Basil L. Contovounesios]

Stefan Monnier [2021-09-27 14:38 -0400] wrote:

> Basil L. Contovounesios [2021-09-26 17:49:33] wrote:
>> Stefan Monnier [2021-09-24 09:14 -0400] wrote:
>>> Can you guys come up with an idea of how to change shortdoc.el so that
>>> your package can do its job more easily and cleanly?
>>> [ Ideally without having to use eval-after-load nor having to require
>>> shortdoc.  ]
>>
>> Apart from eval-after-load, what other mechanisms do we have that can
>> register information before a package is loaded?  There's obarray and
>> there are hooks, is there something else?
>
> For buffer-local info, the standard way is for the amjor mode to do
>
>     (setq-local <foo> <bar>)
>
> Example of <foo> include `font-lock-defaults` (back when font-lock was
> not enabled by default), `outline-regexp`, `imenu-generic-expression`, ...
>
> Here I think we want something that's not buffer-local, so we'd need
> something else.  Not sure what's the best option.  Maybe
>
>     (add-hook 'shortdoc-foo-functions #'...)
>
> ?

It faces the same problem: shortdoc somehow needs to know about 'foo'
a priori, for interactive group name completion.

>> The entrypoint command shortdoc-display-group could, every time it is
>> invoked, scan obarray for certain symbol properties, or run a hook, to
>> accumulate auxiliary shortdoc forms, in addition to those that were
>> already permanently stored in shortdoc--groups.
>
> That's also an option, yes (scanning `obarray` is fairly quick).

I'll try to propose something soon™ unless someone beats me to it.

Thanks,

-- 
Basil

Re: Supporting newer Emacs features in older Emacs versions without warnings?

[Stefan Monnier]

>> Here I think we want something that's not buffer-local, so we'd need
>> something else.  Not sure what's the best option.  Maybe
>>
>>     (add-hook 'shortdoc-foo-functions #'...)
>>
>> ?
>
> It faces the same problem: shortdoc somehow needs to know about 'foo'
> a priori, for interactive group name completion.

No, I meant for `foo` to be something like "extra" (i.e. a fixed name
that I just didn't feel like choosing at the time I wrote the example).
I.e. there'd be one shortdoc hook onto which packages would hang
their functions that return extra entries.  So instead of iterating through
the obarray you'd only iterate through the list of functions on
that hook.


        Stefan

Re: Supporting newer Emacs features in older Emacs versions without warnings?

[Basil L. Contovounesios]

Stefan Monnier [2021-09-28 08:51 -0400] wrote:

>>> Here I think we want something that's not buffer-local, so we'd need
>>> something else.  Not sure what's the best option.  Maybe
>>>
>>>     (add-hook 'shortdoc-foo-functions #'...)
>>>
>>> ?
>>
>> It faces the same problem: shortdoc somehow needs to know about 'foo'
>> a priori, for interactive group name completion.
>
> No, I meant for `foo` to be something like "extra" (i.e. a fixed name
> that I just didn't feel like choosing at the time I wrote the example).
> I.e. there'd be one shortdoc hook onto which packages would hang
> their functions that return extra entries.  So instead of iterating through
> the obarray you'd only iterate through the list of functions on
> that hook.

Ah, right, then that's indeed one of the two options I was originally
thinking of (the other being global symbol properties).

-- 
Basil