custom-reevaluate-setting / custom-initialize-delay

custom-reevaluate-setting / custom-initialize-delay

[David Reitter]

There's a bug in the initialization customization variables when this initialization is delayed using `custom-initialize-delay' function.

These get init'd during dumping rather than at run-time.  The documentation suggests that it's supposed to happen at run-time.

A symptom of this is that `send-mail-function' is always set to `sendmail-send-it', because `window-system' is not available during dumping.
This is a regression compared to Emacs 22, where this mechanism worked.

I think the correct fix is to not clear `custom-delayed-init-variables' while dumping.  Possibly, one would want to not initialize the variables at all (see also the comment in `custom-initialize-delay' w.r.t. evaluating value).   The patch below just implements the variant with the smaller change in behavior.


diff --git a/lisp/startup.el b/lisp/startup.el
index 87f1a00..0c9b190 100644
--- a/lisp/startup.el
+++ b/lisp/startup.el
@@ -913,10 +913,12 @@ opening the first frame (e.g. open a connection to an X server).")
   ;; Re-evaluate predefined variables whose initial value depends on
   ;; the runtime context.
   (mapc 'custom-reevaluate-setting
-        ;; Initialize them in the same order they were loaded, in case there
-        ;; are dependencies between them.
-        (prog1 (nreverse custom-delayed-init-variables)
-          (setq custom-delayed-init-variables nil)))
+	;; Initialize them in the same order they were loaded, in case there
+	;; are dependencies between them.
+	(prog1 (nreverse custom-delayed-init-variables)
+	    (unless (or (member (nth 3 command-line-args) '("dump" "bootstrap"))
+	      (member (nth 4 command-line-args) '("dump" "bootstrap")))
+	      (setq custom-delayed-init-variables nil))))
 
   (normal-erase-is-backspace-setup-frame)
 

Re: custom-reevaluate-setting / custom-initialize-delay

[Glenn Morris]

David Reitter wrote:

> There's a bug

Bugs reported to the bug list are much less likely to be
forgotten/overlooked...

>  in the initialization customization variables when this
>  initialization is delayed using `custom-initialize-delay' function.
>
> These get init'd during dumping rather than at run-time.

This can't be the whole story, because this works (current trunk and
23.1 on GNU/Linux):

emacs -Q
C-h v temporary-file-directory     -> "/tmp/"

TMPDIR=/var/tmp/ emacs -Q
C-h v temporary-file-directory     -> "/var/tmp/"

> A symptom of this is that `send-mail-function' is always set to
> `sendmail-send-it', because `window-system' is not available during
> dumping.

I think this is specific to send-mail-function, as the only (?) user
of custom-initialize-delay where the defining file is not preloaded,
but rather the defcustom is autoloaded. There is already some attempt
at a manual workaround for that in sendmail.el, but I guess it doesn't
work. If the definition of send-mail-function is moved to startup.el,
it works fine.


send-mail-function's default value probably shouldn't be based on
window-system anyway, now that X and tty frames can be mixed (though I
don't know if this works on Windows/Nextstep). I think
mailclient-send-it itself should check window-system and call
sendmail-send-it if it is nil.

Re: custom-reevaluate-setting / custom-initialize-delay

[David Reitter]

[-- Attachment #1: Type: text/plain, Size: 1338 bytes --]

Yes, send-mail-function is the only delayed-init defcustom that's autoloaded.

On Feb 23, 2010, at 1:04 PM, Glenn Morris wrote:

> send-mail-function's default value probably shouldn't be based on
> window-system anyway, now that X and tty frames can be mixed (though I
> don't know if this works on Windows/Nextstep). I think
> mailclient-send-it itself should check window-system and call
> sendmail-send-it if it is nil.

I don't think that this would be a user-friendly solution. 
It should be `initial-window-system', not `window-system', however.  I find that `browse-url' is not useful on remote emacs sessions via SSH.

You may recall that the default value is different on GNU/Linux because we expect users to configure their Emacs mail system explicitly (e.g., with a working sender address), or that the `browse-url' (with a mailto://) mechanism is less likely to produce the desired effects (haven't tested this).   Also, an older OS X version didn't correctly run `sendmail'.

The underlying reason for all of this is independent of the system - firewalls block outgoing mail traffic, and mail servers refuse to deliver mail from dial-up IPs.
I noticed all of this because a user complained that his bug reports didn't go through (but were silently swallowed).

And no, X/TTY mixing doesn't work on NS. 


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 203 bytes --]

Re: custom-reevaluate-setting / custom-initialize-delay

[Glenn Morris]

David Reitter wrote:

> And no, X/TTY mixing doesn't work on NS. 

Then it isn't an issue. But if it is ever implemented and people start
using eg emacs --daemon, send-mail-function will be incorrectly
initialized.

Anyway, the following seems to work:


*** lisp/mail/sendmail.el   2010-01-17 23:34:53 +0000
--- lisp/mail/sendmail.el   2010-02-24 01:13:01 +0000
***************
*** 170,175 ****
--- 170,177 ----
    :initialize 'custom-initialize-delay
    :group 'sendmail)
  
+ ;;;###autoload(custom-initialize-delay 'send-mail-function nil)
+ 
  ;;;###autoload
  (defcustom mail-header-separator (purecopy "--text follows this line--")
    "Line used to separate headers from text in messages being composed."

Re: custom-reevaluate-setting / custom-initialize-delay

[David Reitter]

[-- Attachment #1: Type: text/plain, Size: 778 bytes --]

On Feb 23, 2010, at 8:14 PM, Glenn Morris wrote:
> + ;;;###autoload(custom-initialize-delay 'send-mail-function nil)
> + 
>  ;;;###autoload
>  (defcustom mail-header-separator (purecopy "--text follows this line--")

That's a smaller change and maybe the right thing to do right now, but I'd still like to understand why you accept the initialization mechanism in startup.el to init only at dump time and not at run-time in the case of send-mail-function. That's what seemed to be the real problem.  Adding code to loaddefs.el seems like a workaround.

Btw. I considered a solution like yours, but I didn't figure you could do this sort of thing with the ###autoload cookies.  I thought, from a comment in autoload.el, that they were restricted to defcustoms and defuns.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 203 bytes --]

Re: custom-reevaluate-setting / custom-initialize-delay

[Glenn Morris]

Glenn Morris wrote:

> + ;;;###autoload(custom-initialize-delay 'send-mail-function nil)

I installed that because I think something like this is important for
tomorrow's (?) pretest. Anyone is of course welcome to change it.

Re: custom-reevaluate-setting / custom-initialize-delay

[Eli Zaretskii]

> From: Glenn Morris <rgm@gnu.org>
> Date: Tue, 23 Feb 2010 13:04:33 -0500
> Cc: emacs-devel@gnu.org
> 
> send-mail-function's default value probably shouldn't be based on
> window-system anyway, now that X and tty frames can be mixed (though I
> don't know if this works on Windows/Nextstep).

On Windows, it doesn't currently work to mix tty and GUI frames in the
same session (because no one has figured out how to do that).

Re: custom-reevaluate-setting / custom-initialize-delay

[Eli Zaretskii]

> From: Glenn Morris <rgm@gnu.org>
> Date: Tue, 23 Feb 2010 13:04:33 -0500
> Cc: emacs-devel@gnu.org
> 
> send-mail-function's default value probably shouldn't be based on
> window-system anyway, now that X and tty frames can be mixed (though I
> don't know if this works on Windows/Nextstep). I think
> mailclient-send-it itself should check window-system and call
> sendmail-send-it if it is nil.

Could this be the reason for Bug#5299, per chance?

Drew, does the problem in #5299 go away for you if you set
send-mail-function to mailclient-send-it?

RE: custom-reevaluate-setting / custom-initialize-delay

[Drew Adams]

> > send-mail-function's default value probably shouldn't be based on
> > window-system anyway, now that X and tty frames can be 
> > mixed (though I don't know if this works on Windows/Nextstep). I think
> > mailclient-send-it itself should check window-system and call
> > sendmail-send-it if it is nil.
> 
> Could this be the reason for Bug#5299, per chance?
> 
> Drew, does the problem in #5299 go away for you if you set
> send-mail-function to mailclient-send-it?

Nope (just tried it).

As Lennart and I have said, there was some Windows-specific code that did this:

. When you hit C-c C-c, it popped up the mail client (in my case, Outlook) with
a new message window.

. The message was pre-address to the right bug list (pretest or bugs). It had
the Subject filled out.

. But the message body did not have the body that you entered in Emacs. Instead,
it had this text (still does, for Emacs 22):

  *** E-Mail body has been placed on clipboard, please paste them here! *** 

All you had to do then was use `C-v' to paste the body you wrote in place of
that "***...***" text.

HTH - Drew

P.S. I barely noticed your question to me. I haven't been reading this thread.

Re: custom-reevaluate-setting / custom-initialize-delay

[Eli Zaretskii]

> From: "Drew Adams" <drew.adams@oracle.com>
> Cc: <david.reitter@gmail.com>, <emacs-devel@gnu.org>
> Date: Tue, 23 Feb 2010 11:08:13 -0800
> 
> > Drew, does the problem in #5299 go away for you if you set
> > send-mail-function to mailclient-send-it?
> 
> Nope (just tried it).

How did it fail?  What happened?

> As Lennart and I have said, there was some Windows-specific code that did this:
> 
> . When you hit C-c C-c, it popped up the mail client (in my case, Outlook) with
> a new message window.
> 
> . The message was pre-address to the right bug list (pretest or bugs). It had
> the Subject filled out.
> 
> . But the message body did not have the body that you entered in Emacs. Instead,
> it had this text (still does, for Emacs 22):
> 
>   *** E-Mail body has been placed on clipboard, please paste them here! *** 
> 
> All you had to do then was use `C-v' to paste the body you wrote in place of
> that "***...***" text.

That's what mailclient-send-it is supposed to do, AFAIK.

RE: custom-reevaluate-setting / custom-initialize-delay

[Drew Adams]

> > > Drew, does the problem in #5299 go away for you if you set
> > > send-mail-function to mailclient-send-it?
> > 
> > Nope (just tried it).
> 
> How did it fail?  What happened?

See the bug report or the other message I just sent replying to this thread.

> > As Lennart and I have said, there was some Windows-specific 
> > code that did this:
> > 
> > . When you hit C-c C-c, it popped up the mail client (in my 
> >   case, Outlook) with a new message window.
> > 
> > . The message was pre-address to the right bug list 
> >   (pretest or bugs). It had the Subject filled out.
> > 
> > . But the message body did not have the body that you 
> >   entered in Emacs. Instead, it had this text (still does,
> >   for Emacs 22):
> > 
> >   *** E-Mail body has been placed on clipboard, please 
> >   paste them here! *** 
> > 
> > All you had to do then was use `C-v' to paste the body you 
> > wrote in place of that "***...***" text.
> 
> That's what mailclient-send-it is supposed to do, AFAIK.

Well something has clearly been broken since 23.1.

Re: custom-reevaluate-setting / custom-initialize-delay

[Glenn Morris]

Eli Zaretskii wrote:

> Drew, does the problem in #5299 go away for you if you set
> send-mail-function to mailclient-send-it?

It's message-send-mail-function (the variable) that one needs to set
nowadays.

The function message-send-mail-function may also need some adjustment
for Windows and Macs to prefer mailclient more strongly than it
currently does.

Re: custom-reevaluate-setting / custom-initialize-delay

[Eli Zaretskii]

> Cc: david.reitter@gmail.com,  emacs-devel@gnu.org
> From: Glenn Morris <rgm@gnu.org>
> Date: Tue, 23 Feb 2010 22:09:32 -0500
> 
> Eli Zaretskii wrote:
> 
> > Drew, does the problem in #5299 go away for you if you set
> > send-mail-function to mailclient-send-it?
> 
> It's message-send-mail-function (the variable) that one needs to set
> nowadays.

Drew, could you please try that and tell the results?

RE: custom-reevaluate-setting / custom-initialize-delay

[Drew Adams]

> > Drew, does the problem in #5299 go away for you if you set
> > send-mail-function to mailclient-send-it?
> 
> It's message-send-mail-function (the variable) that one needs to set
> nowadays.
> 
> The function message-send-mail-function may also need some adjustment
> for Windows and Macs to prefer mailclient more strongly than it
> currently does.

OK, I tried that too:
(setq message-send-mail-function 'mailclient-send-it)

Same thing:

shell-command-to-string: Searching for program: no such file or directory,
/bin/bash

This is nothing like what it used to do, sadly. Just try Emacs 22.[123] or 23.1
on Windows, `emacs -Q', no changes. Works fine.

Re: custom-reevaluate-setting / custom-initialize-delay

[Reiner Steib]

On Wed, Feb 24 2010, Drew Adams wrote:

> OK, I tried that too:
> (setq message-send-mail-function 'mailclient-send-it)

I guess this should read ...
(setq message-send-mail-function 'message-send-mail-with-mailclient)

,----[ <f1> v message-send-mail-function RET ]
| message-send-mail-function is a variable defined in `message.el'.
| Its value is 
| message-send-mail-with-sendmail
| 
| Documentation:
| Function to call to send the current buffer as mail.
| The headers should be delimited by a line whose contents match the
| variable `mail-header-separator'.
| 
| Valid values include `message-send-mail-with-sendmail'
| `message-send-mail-with-mh', `message-send-mail-with-qmail',
| `message-smtpmail-send-it', `smtpmail-send-it',
| `feedmail-send-it' and `message-send-mail-with-mailclient'.  The
| default is system dependent and determined by the function
| `message-send-mail-function'.
| 
| See also `send-mail-function'.
| 
| You can customize this variable.
`----

Bye, Reiner.
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo---  |  PGP key available  |  http://rsteib.home.pages.de/

RE: custom-reevaluate-setting / custom-initialize-delay

[Drew Adams]

> > >>> Drew, does the problem in #5299 go away for you if you set
> > >>> send-mail-function to mailclient-send-it?
> > >>
> > >> It's message-send-mail-function (the variable) that
> > >> one needs to set nowadays.
> > >
> > > OK, I tried that too:
> > > (setq message-send-mail-function 'mailclient-send-it)
> 
> I guess this should read ...
> (setq message-send-mail-function
>       'message-send-mail-with-mailclient)

Dunno what you mean by "should read". It wasn't a typo.
I just did what was suggested.

Maybe you mean "Try this instead"?

Re: custom-reevaluate-setting / custom-initialize-delay

[Eli Zaretskii]

> From: "Drew Adams" <drew.adams@oracle.com>
> Date: Wed, 24 Feb 2010 08:29:26 -0800
> Cc: emacs-devel@gnu.org
> 
> > > >>> Drew, does the problem in #5299 go away for you if you set
> > > >>> send-mail-function to mailclient-send-it?
> > > >>
> > > >> It's message-send-mail-function (the variable) that
> > > >> one needs to set nowadays.
> > > >
> > > > OK, I tried that too:
> > > > (setq message-send-mail-function 'mailclient-send-it)
> > 
> > I guess this should read ...
> > (setq message-send-mail-function
> >       'message-send-mail-with-mailclient)
> 
> Dunno what you mean by "should read". It wasn't a typo.
> I just did what was suggested.
> 
> Maybe you mean "Try this instead"?

Yes, please try that.  Since message-mode is the default now, it is
important to know that using message-send-mail-with-mailclient will
work for sending bug reports on Windows.

Thanks.

RE: custom-reevaluate-setting / custom-initialize-delay

[Drew Adams]

> > > I guess this should read ...
> > > (setq message-send-mail-function
> > >       'message-send-mail-with-mailclient)
>
> Yes, please try that.  Since message-mode is the default now, it is
> important to know that using message-send-mail-with-mailclient will
> work for sending bug reports on Windows.

No, that doesn't work. Same symptom as before:
"Searching for program: no such file or directory, /bin/bash"

Re: custom-reevaluate-setting / custom-initialize-delay

[Stefan Monnier]

>> > > I guess this should read ...
>> > > (setq message-send-mail-function
>> > >       'message-send-mail-with-mailclient)
>> 
>> Yes, please try that.  Since message-mode is the default now, it is
>> important to know that using message-send-mail-with-mailclient will
>> work for sending bug reports on Windows.

> No, that doesn't work. Same symptom as before:
> "Searching for program: no such file or directory, /bin/bash"

Please send us the backtrace as well (it's generally a good idea to
include a backtrace when you get an error).  Actually, I'd recommend you
run your Emacs with debug-on-error always set to t.  It will
occasionally bring up a backtrace for a "minor error from the user", but
it should be rare.


        Stefan

RE: custom-reevaluate-setting / custom-initialize-delay

[Drew Adams]

> >> > > I guess this should read ...
> >> > > (setq message-send-mail-function
> >> > >       'message-send-mail-with-mailclient)
> >> 
> >> Yes, please try that.  Since message-mode is the default now, it is
> >> important to know that using message-send-mail-with-mailclient will
> >> work for sending bug reports on Windows.
> 
> > No, that doesn't work. Same symptom as before:
> > "Searching for program: no such file or directory, /bin/bash"
> 
> Please send us the backtrace as well (it's generally a good idea to
> include a backtrace when you get an error).

Debugger entered--Lisp error: (file-error "Searching for program" "no such file
or directory" "/bin/bash")
  call-process("/bin/bash" nil t nil "-c" "uncompface")
  shell-command-to-string("uncompface")
  (string-match "^0x" (shell-command-to-string "uncompface"))
  (and (string-match "^0x" (shell-command-to-string "uncompface"))
(executable-find "icontopbm"))
  (if (featurep (quote xemacs)) (featurep (quote xface)) (and (string-match
"^0x" ...) (executable-find "icontopbm")))
  (and (not noninteractive) (gnus-image-type-available-p (quote xbm)) (if
(featurep ...) (featurep ...) (and ... ...)) (quote head))
  eval((and (not noninteractive) (gnus-image-type-available-p (quote xbm)) (if
(featurep ...) (featurep ...) (and ... ...)) (quote head)))
  custom-initialize-reset(gnus-treat-display-x-face (and (not noninteractive)
(gnus-image-type-available-p (quote xbm)) (if (featurep ...) (featurep ...) (and
... ...)) (quote head)))
  custom-declare-variable(gnus-treat-display-x-face (and (not noninteractive)
(gnus-image-type-available-p (quote xbm)) (if (featurep ...) (featurep ...) (and
... ...)) (quote head)) ("c:/Emacs-23-1-92/lisp/gnus/gnus-art.elc" . 52832)
:group gnus-article-treat :version "21.1" :link (custom-manual
"(gnus)Customizing Articles") :link (custom-manual "(gnus)X-Face") :type (choice
(const :tag "Off" nil) (const :tag "Header" head)) :set #[(symbol value)
"\302\b\303\b!\204\x0e

Re: custom-reevaluate-setting / custom-initialize-delay

[Stefan Monnier]

>> >> > > I guess this should read ...
>> >> > > (setq message-send-mail-function
>> >> > >       'message-send-mail-with-mailclient)
>> >> 
>> >> Yes, please try that.  Since message-mode is the default now, it is
>> >> important to know that using message-send-mail-with-mailclient will
>> >> work for sending bug reports on Windows.
>> 
>> > No, that doesn't work. Same symptom as before:
>> > "Searching for program: no such file or directory, /bin/bash"
>> 
>> Please send us the backtrace as well (it's generally a good idea to
>> include a backtrace when you get an error).

> Debugger entered--Lisp error: (file-error "Searching for program" "no such file
> or directory" "/bin/bash")
>   call-process("/bin/bash" nil t nil "-c" "uncompface")
>   shell-command-to-string("uncompface")
>   (string-match "^0x" (shell-command-to-string "uncompface"))

You're using the old gnus-art.el without the patch I sent (and you
tested) earlier.


        Stefan

RE: custom-reevaluate-setting / custom-initialize-delay

[Drew Adams]

> You're using the old gnus-art.el without the patch I sent (and you
> tested) earlier.

Loading a patched gnus-art.el plus setting message-send-mail-function to
message-send-mail-with-mailclient works.

Re: custom-reevaluate-setting / custom-initialize-delay

[Eli Zaretskii]

> Cc: david.reitter@gmail.com,  emacs-devel@gnu.org
> From: Glenn Morris <rgm@gnu.org>
> Date: Tue, 23 Feb 2010 22:09:32 -0500
> 
> Eli Zaretskii wrote:
> 
> > Drew, does the problem in #5299 go away for you if you set
> > send-mail-function to mailclient-send-it?
> 
> It's message-send-mail-function (the variable) that one needs to set
> nowadays.

Drew, could you please try that and tell the results?

Alternatively, set mail-user-agent to sendmail-user-agent and _then_
set send-mail-function to mailclient-send-it.  I think the latter
emulates more closely what you had in Emacs 22.

RE: custom-reevaluate-setting / custom-initialize-delay

[Drew Adams]

> > > Drew, does the problem in #5299 go away for you if you set
> > > send-mail-function to mailclient-send-it?
> > 
> > It's message-send-mail-function (the variable) that one needs to set
> > nowadays.
> 
> Drew, could you please try that and tell the results?
> 
> Alternatively, set mail-user-agent to sendmail-user-agent and _then_
> set send-mail-function to mailclient-send-it.  I think the latter
> emulates more closely what you had in Emacs 22.

We have a winner!

Yep, that's the ticket. It restores the Emacs 23.1 (and Emacs 22) behavior.

Thx - Drew

RE: custom-reevaluate-setting / custom-initialize-delay

[Drew Adams]

> Yep, that's the ticket. It restores the Emacs 23.1 (and Emacs 
> 22) behavior.

(Needless to say, I hope, that this should be the out-of-the-box behavior, as
before. Users should not need to set such variables themselves.)

Re: custom-reevaluate-setting / custom-initialize-delay

[Lennart Borgman]

On Wed, Feb 24, 2010 at 5:26 AM, Drew Adams <drew.adams@oracle.com> wrote:
>> Yep, that's the ticket. It restores the Emacs 23.1 (and Emacs
>> 22) behavior.
>
> (Needless to say, I hope, that this should be the out-of-the-box behavior, as
> before. Users should not need to set such variables themselves.)


Of course sending bug reports should work out-of-the-box.

If we ever get a test suite Emacs this is the first thing that should be tested.

Re: custom-reevaluate-setting / custom-initialize-delay

[Stefan Monnier]

> There's a bug in the initialization customization variables when this
> initialization is delayed using `custom-initialize-delay' function.

> These get init'd during dumping rather than at run-time.
> The documentation suggests that it's supposed to happen at run-time.

The problem is for customization variables which use
custom-initialize-delay and are autoloaded, because autoload.el ends up
stripping away the custom-initialize-delay.

So either we should fix autoload.el to handle this correctly or we
should not use ;;;###autoload on such defcustom.


        Stefan

Re: custom-reevaluate-setting / custom-initialize-delay

[Glenn Morris]

Stefan Monnier wrote:

> The problem is for customization variables which use
> custom-initialize-delay and are autoloaded, because autoload.el ends up
> stripping away the custom-initialize-delay.
>
> So either we should fix autoload.el to handle this correctly or we
> should not use ;;;###autoload on such defcustom.

It's up to you of course, but IMO since this only affects one
variable, just fixing it "by hand" with a manual autoload of
custom-initialize-delay (together with a doc-fix for said function)
seems fine for 23.2. Sendmail.el already manually autoloads the
saved-value property for some years for similar reasons.