defcustom standard value and byte-compilation

defcustom standard value and byte-compilation

[Tassilo Horn]

Hi all,

I just found out that the form defining the standard value of a
defcustom is not subject to byte-compilation.  The problem with that is
that you can't use macros in there without having to require the file
providing the macro at load-time, too.

This is the concrete example in AUCTeX's tex.el:

--8<---------------cut here---------------start------------->8---
(eval-when-compile (require 'cl))

(defcustom TeX-style-path
  (let ((path))
    (mapc (lambda (file) (when file (pushnew file path)))
          (append (list TeX-auto-global TeX-style-global)
                  TeX-auto-private TeX-style-private
                  (list TeX-auto-local TeX-style-local)))
    (nreverse path))
  ;; docs & keywords
  )
--8<---------------cut here---------------end--------------->8---

In the byte-compiled tex.elc, the reference to pushnew is still there
and you get an error when loading tex.el and cl hasn't been required
beforehand.

I fixed this concrete situation by using `member' and `cons' instead of
`pushnew' but IMHO this behavior is at the very least surprising.  I can
use macros in functions and init-forms of defvars and defconsts and they
are all compiled away, so why shouldn't I allowed to do the same for
defcustoms?

Also, I don't find a note on that behavior in the docs.

Bye,
Tassilo

Re: defcustom standard value and byte-compilation

[Stefan Monnier]

> I just found out that the form defining the standard value of a
> defcustom is not subject to byte-compilation.

IIRC this is not the case if you use lexical-binding.

> The problem with that is that you can't use macros in there without
> having to require the file providing the macro at load-time, too.

Indeed, and the byte-compiler won't warn about unknown functions/vars
and things like that.

> I can use macros in functions and init-forms of defvars and defconsts
> and they are all compiled away, so why shouldn't I allowed to do the
> same for defcustoms?

IIIRC there's a pending bug report about the new behavior in
lexical-binding mode, because when you M-x customize-variable and ask to
see the expression it shows you the byte-compiled gobbledygook instead
of a readable expression.

> Also, I don't find a note on that behavior in the docs.

I don't think it was a conscious decision.


        Stefan

Re: defcustom standard value and byte-compilation

[Tassilo Horn]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> I just found out that the form defining the standard value of a
>> defcustom is not subject to byte-compilation.
>
> IIRC this is not the case if you use lexical-binding.

I don't think this is an option for AUCTeX as it supports Emacs versions
down to 21.4 and also XEmacs.  And several parts make use of the dynamic
binding of locals. :-(

>> The problem with that is that you can't use macros in there without
>> having to require the file providing the macro at load-time, too.
>
> Indeed, and the byte-compiler won't warn about unknown functions/vars
> and things like that.
>
>> I can use macros in functions and init-forms of defvars and defconsts
>> and they are all compiled away, so why shouldn't I allowed to do the
>> same for defcustoms?
>
> IIIRC there's a pending bug report about the new behavior in
> lexical-binding mode, because when you M-x customize-variable and ask
> to see the expression it shows you the byte-compiled gobbledygook
> instead of a readable expression.

But customize shows the value of the expression, not the expression
itself.  So in my case the value is a list of strings which should not
be different no matter if it has been computed by a byte-compiled
expression or a non-b-c'ed expression.

Ah, I think I got it.  The difference is when the standard expression
computes a value which can be byte-compiled, e.g., a function.  If the
standard expression is byte-compiled, it'll result in a byte-compiled
function which is not readable anymore.

Ok, but couldn't the standard form be at least macroexpanded?  That
shouldn't cause any harm and would take care of the typical use-case
where one uses macros of a package which might not be available at
load-time.

>> Also, I don't find a note on that behavior in the docs.
>
> I don't think it was a conscious decision.

Probably.  And at least it's consistent with older Emacs versions and
XEmacs.

Bye,
Tassilo

Re: defcustom standard value and byte-compilation

[Stefan Monnier]

> I don't think this is an option for AUCTeX as it supports Emacs versions
> down to 21.4 and also XEmacs.

Of course, I think that's a mistake ;-)

> And several parts make use of the dynamic binding of locals. :-(

That's no problem, lexical-binding supports dynamically scoped variables
as well (you just have to declare them beforehand via (defvar <var>)).

> But customize shows the value of the expression, not the expression
> itself.

IIRC there's a way to get the expression rather than the value.

> Ok, but couldn't the standard form be at least macroexpanded?  That
> shouldn't cause any harm and would take care of the typical use-case
> where one uses macros of a package which might not be available at
> load-time.

Changing the defcustom behavior in Emacs-25.1 to be the same in
dynamic-binding than with lexical-binding wouldn't help you with XEmacs
and Emacs-21.4.


        Stefan

Re: defcustom standard value and byte-compilation

[Tassilo Horn]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> I don't think this is an option for AUCTeX as it supports Emacs
>> versions down to 21.4 and also XEmacs.
>
> Of course, I think that's a mistake ;-)

I'd have no problems cutting support for emacs <23 or even <24, but
cutting support for XEmacs is something which I'd prefer not to do
without extremely good reasons although that would make life much more
easy.

>> And several parts make use of the dynamic binding of locals. :-(
>
> That's no problem, lexical-binding supports dynamically scoped
> variables as well (you just have to declare them beforehand via
> (defvar <var>)).

Yes, I know.  But having non-prefixed (defvar name) etc just looks so
damn wrong.

>> But customize shows the value of the expression, not the expression
>> itself.
>
> IIRC there's a way to get the expression rather than the value.

At least I couldn't find one in the customize UI but maybe you're right.

>> Ok, but couldn't the standard form be at least macroexpanded?  That
>> shouldn't cause any harm and would take care of the typical use-case
>> where one uses macros of a package which might not be available at
>> load-time.
>
> Changing the defcustom behavior in Emacs-25.1 to be the same in
> dynamic-binding than with lexical-binding wouldn't help you with
> XEmacs and Emacs-21.4.

No, sure.  And macroexpanding in dynamic-binding wouldn't make it the
same with byte-compilation in lexical-binding.  But still it would
eliminate the (void-function some-macro) error at load-time.  There's
also an issue from 2011 about the very same thing:

  https://debbugs.gnu.org/cgi/bugreport.cgi?bug=9712

But I can also see the good thing about the current behavior.  It
consistently fails on all Emacs/XEmacs versions (if dynamic-binding is
in use and not something else loaded, e.g., cl, before the file in
question).

Bye,
Tassilo

Re: defcustom standard value and byte-compilation

[Stefan Monnier]

>> That's no problem, lexical-binding supports dynamically scoped
>> variables as well (you just have to declare them beforehand via
>> (defvar <var>)).
> Yes, I know.  But having non-prefixed (defvar name) etc just looks so
> damn wrong.

Not having the (defvar <foo>) doesn't make any difference: the code
still uses those non-prefixed names.

>>> But customize shows the value of the expression, not the expression
>>> itself.
>> IIRC there's a way to get the expression rather than the value.
> At least I couldn't find one in the customize UI but maybe you're right.

I can't find it either.  And I can't even remember where the bug was
reported.  Maybe the bug was slightly different than what I said, I just
have a faint memory of it.


        Stefan

RE: defcustom standard value and byte-compilation

[Drew Adams]

> >>> But customize shows the value of the expression, not the expression
> >>> itself.
> >> IIRC there's a way to get the expression rather than the value.
> > At least I couldn't find one in the customize UI but maybe you're right.
> 
> I can't find it either.  And I can't even remember where the bug was
> reported.  Maybe the bug was slightly different than what I said, I just
> have a faint memory of it.

I haven't been following this.  But are you perhaps looking for
State > Show Saved Lisp Expression?

Re: defcustom standard value and byte-compilation

[Stephen J. Turnbull]

Tassilo Horn writes:

 > I'd have no problems cutting support for emacs <23 or even <24, but
 > cutting support for XEmacs is something which I'd prefer not to do
 > without extremely good reasons although that would make life much more
 > easy.

Greatly appreciated!

BTW, at the time I write this, Uwe hasn't responded to what I told him
on xemacs-beta, but I'm pretty sure his problem is that he loads
tex.el (from AUCTeX) in his init file.  By default XEmacs loads
customizations *after* init[1], so if Uwe used custom to set those
variables, they were empty when the defcustom was encountered in
tex.el.

 > Yes, I know.  But having non-prefixed (defvar name) etc just looks
 > so damn wrong.

If these aren't used to communicate with non-AUCTeX applications, just
rename them.  (I'm not sure if that is a wisecrack or a serious
suggestion.)

Footnotes: 
[1]  I think since 21.4.  Why?  There was a convincing use case for
computing things that Customize depended on (perhaps the customization
file's name), don't recall the details but everybody in XEmacs-Review
was at least +0.  It's trivial to load custom.el by hand in init with
this order, to inhibit early loading requires a command-line option.

Re: defcustom standard value and byte-compilation

[Tassilo Horn]

"Stephen J. Turnbull" <stephen@xemacs.org> writes:

>  > I'd have no problems cutting support for emacs <23 or even <24, but
>  > cutting support for XEmacs is something which I'd prefer not to do
>  > without extremely good reasons although that would make life much
>  > more easy.
>
> Greatly appreciated!

You're welcome.

> BTW, at the time I write this, Uwe hasn't responded to what I told him
> on xemacs-beta, but I'm pretty sure his problem is that he loads
> tex.el (from AUCTeX) in his init file.

I'm not sure to which of Uwe's problems you are referring to.  He's had
tons of them but I think we resolved most if not all of them in the
meantime.  (Most were configuration issues on his side.)

> By default XEmacs loads customizations *after* init[1], so if Uwe used
> custom to set those variables, they were empty when the defcustom was
> encountered in tex.el.

Ah, yes, I think he had also a problem with TeX-style-path having a
non-standard value.  Maybe this has something to with the above although
I think he had that when he used GNU Emacs for testing and insisting not
to use one of the documented ways of installing AUCTeX.

>  > Yes, I know.  But having non-prefixed (defvar name) etc just looks
>  > so damn wrong.
>
> If these aren't used to communicate with non-AUCTeX applications, just
> rename them.  (I'm not sure if that is a wisecrack or a serious
> suggestion.)

You can never know.  I think at least reftex makes use of some of them
but that's under our control, too.  And some of them are exposed to user
configurations as well, e.g., entries of `TeX-expand-list' may assume
that `file' is bound to the file to be processed.  There are probably
others, too.

Bye,
Tassilo

Re: defcustom standard value and byte-compilation

[Tassilo Horn]

Drew Adams <drew.adams@oracle.com> writes:

>> >> IIRC there's a way to get the expression rather than the value.
>
> I haven't been following this.  But are you perhaps looking for
> State > Show Saved Lisp Expression?

Hm, the entry's name looks promising but the effect is not.

BEFORE:

--8<---------------cut here---------------start------------->8---
Hide Tex Style Path:
INS DEL /usr/local/var/auctex
INS DEL /home/horn/Repos/el/auctex/style
INS DEL /home/horn/.emacs.d/auctex/auto
INS DEL /home/horn/.emacs.d/auctex/style
INS DEL auto
INS DEL style
INS
    State : STANDARD.
--8<---------------cut here---------------end--------------->8---

AFTER:

--8<---------------cut here---------------start------------->8---
Hide TeX-style-path: 
'("/usr/local/var/auctex" "/home/horn/Repos/el/auctex/style" "/home/horn/.emacs.d/auctex/auto" "/home/horn/.emacs.d/auctex/style" "auto" "style")
    State : STANDARD. (lisp)
--8<---------------cut here---------------end--------------->8---

So that doesn't really show the Lisp expression which computes the
standard value, it just shows the standard value as Lisp expression...

Bye,
Tassilo

Re: defcustom standard value and byte-compilation

[Stefan Monnier]

> I haven't been following this.  But are you perhaps looking for
> State > Show Saved Lisp Expression?

No, This is almost it, but not quite: it doesn't show the *standard* but
the *saved* expression.


        Stefan

RE: defcustom standard value and byte-compilation

[Drew Adams]

> > are you perhaps looking for State > Show Saved Lisp Expression?
> 
> that doesn't really show the Lisp expression which computes the
> standard value, it just shows the standard value as Lisp expression...

Right.  Actually, it shows a Lisp expression which, if evaluated,
returns the standard value.  But it does not show the actual
initial code that produced the standard value.

For example, I have this:

 (defcustom foo (if (> emacs-major-version 23) ; `S-TAB'
                    '([backtab])
                  '([S-tab] [S-iso-lefttab]))

And the menu option shows this as the Lisp expression:

 '([backtab]) ; note the quote

It does not show the standard value, which is ([backtab]).

I'd be surprised if the expression that computes the
initial value is saved anywhere (besides in the source file).

Re: defcustom standard value and byte-compilation

[Andreas Schwab]

Drew Adams <drew.adams@oracle.com> writes:

> I'd be surprised if the expression that computes the
> initial value is saved anywhere (besides in the source file).

(get 'foo 'standard-value)

Andreas.

-- 
Andreas Schwab, SUSE Labs, schwab@suse.de
GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE  1748 E4D4 88E3 0EEA B9D7
"And now for something completely different."

RE: defcustom standard value and byte-compilation

[Drew Adams]

> (get 'foo 'standard-value)

Oops; how could I have forgotten?  (I use it in dozens of places.)