cl-eval-when -- A workaround for recursive require?

cl-eval-when -- A workaround for recursive require?

[Zhu Zihao]

[-- Attachment #1: Type: text/plain, Size: 1399 bytes --]

Hi Emacser!

I invesitgated into the `cl-eval-when` a time long ago about this
`cl-eval-when` hack in the source of magit here.
https://github.com/magit/magit/blob/master/lisp/magit.el#L641

A breif summary: magit.el use a `cl-eval-when` block with load time and
eval time only evaluation to require its sub-components, while each
sub-component use `(require 'magit)` to use procedure in different
sub-components. This hack seems to be a hack to avoid recursive require.

The result of my investigation I remember is: "Don't use cl-eval-when,
it's not robust for many eval strategy combination. eval-when-compile or
eval-and-compile are more reasonable alternatives." But today I see a
thread on Emacs China forum which refered to this hack in magit.el.

First I guess this works because such `require` statements is placed
after `provide`, and it's not related to `cl-eval-when`. For testing, I
remove the `cl-eval-when` surrounding the require statements in
magit.el. But when I compile these files, I see a lot of warning message
about function not defined.

The bytecomp.el file is a monolith and I don't know how to read and
understand it. Curiosity drives me to ask a question here: How does
`cl-eval-when` hack works for recurisve require? Is it a ugly hack?
 
-- 
Retrieve my PGP public key:

  gpg --recv-keys D47A9C8B2AE3905B563D9135BE42B352A9F6821F

Zihao

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 255 bytes --]

Re: cl-eval-when -- A workaround for recursive require?

[Stefan Monnier]

Zhu Zihao [2022-04-27 18:52:07] wrote:
> I investigated into the `cl-eval-when` a time long ago about this
> `cl-eval-when` hack in the source of magit here.
> https://github.com/magit/magit/blob/master/lisp/magit.el#L641
>
> A breif summary: magit.el use a `cl-eval-when` block with load time and
> eval time only evaluation to require its sub-components, while each
> sub-component use `(require 'magit)` to use procedure in different
> sub-components. This hack seems to be a hack to avoid recursive require.

This kind of setup is quite common, but the resulting cyclic
dependencies tend to be a nightmare.

> The result of my investigation I remember is: "Don't use cl-eval-when,
> it's not robust for many eval strategy combination. eval-when-compile or
> eval-and-compile are more reasonable alternatives."

I tend to agree.  But when it comes to circular dependencies like that
of `magit.el` the use of `cl-when-eval` is a minor detail.

> The bytecomp.el file is a monolith and I don't know how to read and
> understand it. Curiosity drives me to ask a question here: How does
> `cl-eval-when` hack works for recurisve require? Is it a ugly hack?

There's nothing special about `cl-eval-when` for cyclic dependencies.
Some parts of `cl-eval-when` rely on ugly hacks, but I don't think it
makes much difference here.

The only sane way to solve these problems is to fix the circular
dependencies, really.

Often one of the best tools for that is to rely on autoloads instead of
`require` for some functions, and the better way to do that is to use an
"internal" autoloads file (i.e. one that's separate from the
`<pkg>-autoloads.el` loaded at startup and that's only loaded when the
package is actually used).

See lisp/**/*-loaddefs.el for examples of such "internal"
autoloads files.

We don't have great support for auto-generation of such files, tho, so
sometimes it requires manual work to set it up (if you search for
`;;;###` in `lisp/Makefile.in` you'll see examples of what I'm
referring to).


        Stefan

Re: cl-eval-when -- A workaround for recursive require?

[Zhu Zihao]

[-- Attachment #1: Type: text/plain, Size: 3391 bytes --]


Hi Stefan, I'm very happy to see your answer!

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> There's nothing special about `cl-eval-when` for cyclic dependencies.
> Some parts of `cl-eval-when` rely on ugly hacks, but I don't think it
> makes much difference here.

Oh, it's worse... sounds that Magit is playing with black magic code. 

I found some design of `cl-eval-when` is not straightforward. I did some
experiment to learn about it.

```
;; bar.el

(require 'cl-lib)

(cl-eval-when (eval)
  (message "I'm running on eval!"))
(cl-eval-when (load)
  (message "I'm running on load!"))
(cl-eval-when (compile)
  (message "I'm running on compile!"))

;; Run `(byte-compile-file "bar.el")`
;;  => I'm running on compile!

;; Run (load-file "bar.el")
;; => I'm running on eval!

;; Open the bar.el file and run `M-x eval-buffer`
;; => I'm running on eval!

;; Run byte compile and `(load-file "bar.elc")`
;; => I'm running on load!

;; Open the byte-compiled file (bar.elc)
;; => (byte-code "\300\301!\210\302\303!\207" [require cl-lib message "I'm running on load!"] 2)

;; Eval the byte-code above
;; => I'm running on load!
```

The `:load-toplevel` means the content of the top-level block will be
available in bytecode. But it doest NOT eval in the plain S-exp file.
And on the contary. It looks weird because IMO `:load-toplevel` is the
timing we call `load` to load Elisp code from file, regardless bytecode
or S-exp file. Is this intened design or a long lasting bug? Because
`info (cl) "Time of Evaluation"` says:

The ‘cl-eval-when’ acts like a ‘progn’ if ‘eval’ is specified, and like
‘nil’ (ignoring the body FORMS) if not.

If only `eval` is specified. the forms will be discarded in
byte-compiled file. Maybe the `(cl-eval-when (load eval) ...)` should be
the equivalent to `progn`.


From the point of my view, `cl-eval-when` sucks and hurts the mental of
code reader and writer. There's many combination for `cl-eval-when` but
only fews are useful for coding, and these usefule combinations can be
replaced by `eval-when-compile` and/or `eval-and-compile`. This two
compiler intrinsics help the code more robust in differnt evaluation
context whether it's byte-compiled or plain S-exp.

I suggest we can mark `cl-eval-when` as deprecated in favour
`eval-when-compile` and/or `eval-and-compile`. 

> Often one of the best tools for that is to rely on autoloads instead of
> `require` for some functions, and the better way to do that is to use an
> "internal" autoloads file (i.e. one that's separate from the
> `<pkg>-autoloads.el` loaded at startup and that's only loaded when the
> package is actually used).
>
> See lisp/**/*-loaddefs.el for examples of such "internal"
> autoloads files.
>
> We don't have great support for auto-generation of such files, tho, so
> sometimes it requires manual work to set it up (if you search for
> `;;;###` in `lisp/Makefile.in` you'll see examples of what I'm
> referring to).

IMO the package.el is quite limited and it may not fit the development
of some Emacs packages (e.g. packages with dynamic module require
compiling, or packages requires preprocessing to locate some resource)
But this is another topic.  

-- 
Retrieve my PGP public key:

  gpg --recv-keys D47A9C8B2AE3905B563D9135BE42B352A9F6821F

Zihao

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 255 bytes --]

Re: cl-eval-when -- A workaround for recursive require?

[Jonas Bernoulli]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> Zhu Zihao [2022-04-27 18:52:07] wrote:
>> A breif summary: magit.el use a `cl-eval-when` block with load time and
>> eval time only evaluation to require its sub-components, while each
>> sub-component use `(require 'magit)` to use procedure in different
>> sub-components. This hack seems to be a hack to avoid recursive require.
>
> This kind of setup is quite common, but the resulting cyclic
> dependencies tend to be a nightmare.

Yeah, I got a bit lazy there, but I am working on it now.

     Jonas

Re: cl-eval-when -- A workaround for recursive require?

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

Should we add some warnings to the documentation of cl-eval-when?

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)