bug#9312: 24.0.50; smtpmail

bug#9312: 24.0.50; smtpmail

[emacs user]

hello, when sending mail using vm during the past two weeks or so, I am
getting the error bellow.  Please let me know if there are any other
details I can provide.  Thanks...

Debugger entered--Lisp error: (wrong-type-argument number-or-marker-p nil)
  byte-code("
  smtpmail-via-smtp((#("xx@yy" 0 4 (fontified nil) 4 14 (fontified
nil) 14 28 (fontified nil))) #<buffer  smtpmail temp>)
  smtpmail-send-it()
  mail-send()
  vm-mail-send()
  vm-mail-send-and-exit(nil)


In GNU Emacs 24.0.50.2 (i386-apple-darwin9.8.0, NS apple-appkit-949.54)
 of 2011-08-17 on xx
Windowing system distributor `Apple', version 10.3.949
configured using `configure  '--with-ns''

bug#9312: 24.0.50; smtpmail

[Glenn Morris]

emacs user wrote:

> Debugger entered--Lisp error: (wrong-type-argument number-or-marker-p nil)
>   byte-code("
>   smtpmail-via-smtp((#("xx@yy" 0 4 (fontified nil) 4 14 (fontified
> nil) 14 28 (fontified nil))) #<buffer  smtpmail temp>)
>   smtpmail-send-it()

This doesn't tell us anything except that something went wrong inside
the smtpmail-via-smtp function. As a first step to get more info, please
try

M-x load-file /path/to/emacs/lisp/mail/smtpmail.el

to load the uncompiled library, set debug-on-error non-nil, repeat the
error and post the resulting, hopefully more detailed, backtrace.

> In GNU Emacs 24.0.50.2 (i386-apple-darwin9.8.0, NS apple-appkit-949.54)
>  of 2011-08-17 on xx

bug#9312: 24.0.50; smtpmail

[Glenn Morris]

emacs user wrote (on Wed, 24 Aug 2011 at 18:10 -0400):

> thanks for the response, please see attached.  apologies for replying
> only to you rather than the group, I couldn't be sure there is no
> private information/ passwords embedded in this output that I am
> attaching.

Fair enough. I am including the list again since I only quote a piece
of what you sent:

> Debugger entered--Lisp error: (wrong-type-argument number-or-marker-p nil)
>   =(nil 530)
>   (and auth-mechanisms (not ask-for-password) (= (car result) 530))

It looks like your smtp server returned something unexpected after
being presented with a "MAIL FROM" command. I made a change to handle
this a little better, but it still probably won't actually work.

The next step is probably to set smtpmail-debug-info non-nil, repeat
the error, and post the contents of the "*trace of SMTP ..." buffer;
perhaps just whatever the server responds after the "MAIL FROM"
command.

bug#9312: 24.0.50; smtpmail

[emacs user]

thanks again, here it is.  note that when I use emacs 23.3 this does work well.

220 HUB00.xx.edu Microsoft ESMTP MAIL Service ready at Wed, 24 Aug
2011 21:53:29 -0400
250-HUB00.xx.edu Hello [10.xx]
250-SIZE 20971520
250-PIPELINING
250-DSN
250-ENHANCEDSTATUSCODES
250-STARTTLS
250-AUTH GSSAPI NTLM
250-8BITMIME
250-BINARYMIME
250 CHUNKING
220 2.0.0 SMTP server ready
250-HUB00.xx.edu Hello [10.xx]
250-SIZE 20971520
250-PIPELINING
250-DSN
250-ENHANCEDSTATUSCODES
250-AUTH GSSAPI NTLM LOGIN
250-8BITMIME
250-BINARYMIME
250 CHUNKING

Process smtpmail connection broken by remote peer
MAIL FROM:<yy@xx.edu> SIZE=365
530 5.7.1 Client was not authenticated
QUIT


On Wed, Aug 24, 2011 at 8:46 PM, Glenn Morris <rgm@gnu.org> wrote:
>
> emacs user wrote (on Wed, 24 Aug 2011 at 18:10 -0400):
>
>> thanks for the response, please see attached.  apologies for replying
>> only to you rather than the group, I couldn't be sure there is no
>> private information/ passwords embedded in this output that I am
>> attaching.
>
> Fair enough. I am including the list again since I only quote a piece
> of what you sent:
>
>> Debugger entered--Lisp error: (wrong-type-argument number-or-marker-p nil)
>>   =(nil 530)
>>   (and auth-mechanisms (not ask-for-password) (= (car result) 530))
>
> It looks like your smtp server returned something unexpected after
> being presented with a "MAIL FROM" command. I made a change to handle
> this a little better, but it still probably won't actually work.
>
> The next step is probably to set smtpmail-debug-info non-nil, repeat
> the error, and post the contents of the "*trace of SMTP ..." buffer;
> perhaps just whatever the server responds after the "MAIL FROM"
> command.
>
>
>

bug#9312: 24.0.50; smtpmail

[Glenn Morris]

emacs user wrote:

> 250 CHUNKING
>
> Process smtpmail connection broken by remote peer
> MAIL FROM:<yy@xx.edu> SIZE=365
> 530 5.7.1 Client was not authenticated
> QUIT

Sorry, I think you need the help of someone who speaks smtp (Lars?).
It looks to me like this is a case (530) that should be handled.
It should try again, but this time asking you for a password
to authenticate to your smtp server. I can't see why it errors out.

bug#9312: 24.0.50; smtpmail

[Lars Magne Ingebrigtsen]

Glenn Morris <rgm@gnu.org> writes:

> Sorry, I think you need the help of someone who speaks smtp (Lars?).
> It looks to me like this is a case (530) that should be handled.
> It should try again, but this time asking you for a password
> to authenticate to your smtp server. I can't see why it errors out.

Did you fix this in revno 105558, but forgot to chose the report?

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

bug#9312: 24.0.50; smtpmail

[Glenn Morris]

Lars Magne Ingebrigtsen wrote:

> Glenn Morris <rgm@gnu.org> writes:
>
>> Sorry, I think you need the help of someone who speaks smtp (Lars?).
>> It looks to me like this is a case (530) that should be handled.
>> It should try again, but this time asking you for a password
>> to authenticate to your smtp server. I can't see why it errors out.
>
> Did you fix this in revno 105558, but forgot to chose the report?

I don't think I fixed it, because I don't see how the debug from

http://debbugs.gnu.org/cgi/bugreport.cgi?bug=9312#16

could lead to the error from the initial report; and the changes in
r105558 just mean Emacs won't throw a wrong-type-argument error, but the
mail still won't get sent, will it?

If the server returns 530, Emacs should try again but prompt for a
password, no?

bug#9312: 24.0.50; smtpmail

[emacs user]

On Sat, Sep 3, 2011 at 3:26 PM, Glenn Morris <rgm@gnu.org> wrote:
> Lars Magne Ingebrigtsen wrote:
>
>> Glenn Morris <rgm@gnu.org> writes:
>>
>>> Sorry, I think you need the help of someone who speaks smtp (Lars?).
>>> It looks to me like this is a case (530) that should be handled.
>>> It should try again, but this time asking you for a password
>>> to authenticate to your smtp server. I can't see why it errors out.
>>
>> Did you fix this in revno 105558, but forgot to chose the report?
>
> I don't think I fixed it, because I don't see how the debug from
>
> http://debbugs.gnu.org/cgi/bugreport.cgi?bug=9312#16
>
> could lead to the error from the initial report; and the changes in
> r105558 just mean Emacs won't throw a wrong-type-argument error, but the
> mail still won't get sent, will it?
>
> If the server returns 530, Emacs should try again but prompt for a
> password, no?
>

I wanted to add that this doesn't happen for all smtp servers.
smtpmail works well for me with google.com, but produces this bug with
what I think is a Microsoft Exchange server.

bug#9312: 24.0.50; smtpmail

[Lars Magne Ingebrigtsen]

Glenn Morris <rgm@gnu.org> writes:

>> 250 CHUNKING
>>
>> Process smtpmail connection broken by remote peer
>> MAIL FROM:<yy@xx.edu> SIZE=365
>> 530 5.7.1 Client was not authenticated
>> QUIT
>
> Sorry, I think you need the help of someone who speaks smtp (Lars?).
> It looks to me like this is a case (530) that should be handled.
> It should try again, but this time asking you for a password
> to authenticate to your smtp server. I can't see why it errors out.

smtpmail.el does the following on MAIL FROM:

	       ((and auth-mechanisms
		     (not ask-for-password)
		     (eq (car result) 530))
		;; We got a "530 auth required", so we close and try
		;; again, this time asking the user for a password.
		(smtpmail-send-command process "QUIT")
		(smtpmail-read-response process)

So it should ask for a password if the server has announced a
willingness to accept auth.

Which it has:

250-AUTH GSSAPI NTLM LOGIN

So I don't understand what's going on here.

Emacs User, could you try to edebug through `smtpmail-via-smtp' and see
what happens when you send an email?

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

bug#9312: 24.0.50; smtpmail

[Lars Magne Ingebrigtsen]

emacs user <user.emacs@gmail.com> writes:

> error: "Process smtpmail not running"

This just means that the SMTP server closed the connection, probably
because you took too long.  Try SPC-ing faster.  :-)

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

bug#9312: 24.0.50; smtpmail

[emacs user]

On Sat, Sep 10, 2011 at 9:43 PM, Lars Magne Ingebrigtsen <larsi@gnus.org> wrote:
>
> smtpmail.el does the following on MAIL FROM:
>
>               ((and auth-mechanisms
>                     (not ask-for-password)
>                     (eq (car result) 530))
>                ;; We got a "530 auth required", so we close and try
>                ;; again, this time asking the user for a password.
>                (smtpmail-send-command process "QUIT")
>                (smtpmail-read-response process)
>
> So it should ask for a password if the server has announced a
> willingness to accept auth.
>
> Which it has:
>
> 250-AUTH GSSAPI NTLM LOGIN
>
> So I don't understand what's going on here.
>
> Emacs User, could you try to edebug through `smtpmail-via-smtp' and see
> what happens when you send an email?

I think the error message comes from this part of the code;
 (smtpmail-send-command
	       process (format "MAIL FROM:<%s>%s%s"
			       envelope-from size-part body-part))

here is some of the edebug output, let me know if there is something
more specific I can provide

Result: (8bitmime (auth gssapi ntlm login) enhancedstatuscodes dsn
(size \20...0))

Result: #<process smtpmail>
 [2 times]
Result: "xx@yy.edu"

Result: " SIZE=373"

Result: ""

Result: "MAIL FROM:<xx@yy.edu> SIZE=373"

error: "Process smtpmail not running"

bug#9312: 24.0.50; smtpmail

[emacs user]

On Mon, Sep 12, 2011 at 8:39 AM, Lars Magne Ingebrigtsen <larsi@gnus.org> wrote:
> emacs user <user.emacs@gmail.com> writes:
>
>> error: "Process smtpmail not running"
>
> This just means that the SMTP server closed the connection, probably
> because you took too long.  Try SPC-ing faster.  :-)

OK, tried again, *much* faster...  same response.  I actually get this
error message also when not using edebug.  here is the traceback.
admittedly, I switched by smtp server away from this one, so am not
affected by this bug personally.  will still be glad to explore this
some more if you feel this may be useful.  cheers, E

Debugger entered--Lisp error: (error "Process smtpmail not running")
  process-send-string(#<process smtpmail> "QUIT")
  smtpmail-send-command(#<process smtpmail> "QUIT")
  byte-code("
  smtpmail-via-smtp((#("xx@yy" 0 4 (fontified nil) 4 14 (fontified
nil) 14 28 (fontified nil))) #<buffer  smtpmail temp>)
  smtpmail-send-it()
  mail-send()
  vm-mail-send()
  vm-mail-send-and-exit(nil)
  call-interactively(vm-mail-send-and-exit nil nil)

bug#9312: 24.0.50; smtpmail

[Lars Magne Ingebrigtsen]

emacs user <user.emacs@gmail.com> writes:

> OK, tried again, *much* faster...  same response.  I actually get this
> error message also when not using edebug.  here is the traceback.
> admittedly, I switched by smtp server away from this one, so am not
> affected by this bug personally.  will still be glad to explore this
> some more if you feel this may be useful.  cheers, E
>
> Debugger entered--Lisp error: (error "Process smtpmail not running")
>   process-send-string(#<process smtpmail> "QUIT")

But now it fails on QUIT and not MAIL FROM, which is progress.  :-)

After sending MAIL FROM, what's the value of the `result' variable?

That is, after calling `smtpmail-read-response' here:

	      (smtpmail-send-command
	       process (format "MAIL FROM:<%s>%s%s"
			       envelope-from size-part body-part))
	      (cond
	       ((smtpmail-ok-p (setq result (smtpmail-read-response process)))
		;; Success.
		)


-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

bug#9312: 24.0.50; smtpmail

[Lars Magne Ingebrigtsen]

emacs user <user.emacs@gmail.com> writes:

>>               ((smtpmail-ok-p (setq result (smtpmail-read-response process)))
>>                ;; Success.
>>                )
>
> is this what we are looking for:
>
> Result: " SIZE=374"
> Result: ""
> Result: "MAIL FROM:<my@address.edu> SIZE=374"
> Result: nil
>  [5 times]
> Result: #<process smtpmail>
> Result: (nil "te peer\nMAIL FROM:<my@address.edu> SIZE=374")
> Result: (nil "te peer\nMAIL FROM:<my@adress.edu> SIZE=374")

Is that the value of the `result' variable?

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

bug#9312: 24.0.50; smtpmail

[emacs user]

On Tue, Sep 13, 2011 at 11:33 PM, Lars Magne Ingebrigtsen
<larsi@gnus.org> wrote:
> emacs user <user.emacs@gmail.com> writes:
>
>> OK, tried again, *much* faster...  same response.  I actually get this
>> error message also when not using edebug.  here is the traceback.
>> admittedly, I switched by smtp server away from this one, so am not
>> affected by this bug personally.  will still be glad to explore this
>> some more if you feel this may be useful.  cheers, E
>>
>> Debugger entered--Lisp error: (error "Process smtpmail not running")
>>   process-send-string(#<process smtpmail> "QUIT")
>
> But now it fails on QUIT and not MAIL FROM, which is progress.  :-)
>
> After sending MAIL FROM, what's the value of the `result' variable?
>
> That is, after calling `smtpmail-read-response' here:
>
>              (smtpmail-send-command
>               process (format "MAIL FROM:<%s>%s%s"
>                               envelope-from size-part body-part))
>              (cond
>               ((smtpmail-ok-p (setq result (smtpmail-read-response process)))
>                ;; Success.
>                )

is this what we are looking for:

Result: " SIZE=374"
Result: ""
Result: "MAIL FROM:<my@address.edu> SIZE=374"
Result: nil
 [5 times]
Result: #<process smtpmail>
Result: (nil "te peer\nMAIL FROM:<my@address.edu> SIZE=374")
Result: (nil "te peer\nMAIL FROM:<my@adress.edu> SIZE=374")

bug#9312: 24.0.50; smtpmail

[emacs user]

On Tue, Sep 13, 2011 at 11:45 PM, Lars Magne Ingebrigtsen
<larsi@gnus.org> wrote:
> emacs user <user.emacs@gmail.com> writes:
>
>>>               ((smtpmail-ok-p (setq result (smtpmail-read-response process)))
>>>                ;; Success.
>>>                )
>>
>> is this what we are looking for:
>>
>> Result: " SIZE=374"
>> Result: ""
>> Result: "MAIL FROM:<my@address.edu> SIZE=374"
>> Result: nil
>>  [5 times]
>> Result: #<process smtpmail>
>> Result: (nil "te peer\nMAIL FROM:<my@address.edu> SIZE=374")
>> Result: (nil "te peer\nMAIL FROM:<my@adress.edu> SIZE=374")
>
> Is that the value of the `result' variable?

I believe it is.  am not too much of an expert...

bug#9312: 24.0.50; smtpmail

[emacs user]

On Tue, Sep 13, 2011 at 11:58 PM, emacs user <user.emacs@gmail.com> wrote:
> On Tue, Sep 13, 2011 at 11:45 PM, Lars Magne Ingebrigtsen
> <larsi@gnus.org> wrote:
>> emacs user <user.emacs@gmail.com> writes:
>>
>>>>               ((smtpmail-ok-p (setq result (smtpmail-read-response process)))
>>>>                ;; Success.
>>>>                )
>>>
>>> is this what we are looking for:
>>>
>>> Result: " SIZE=374"
>>> Result: ""
>>> Result: "MAIL FROM:<my@address.edu> SIZE=374"
>>> Result: nil
>>>  [5 times]
>>> Result: #<process smtpmail>
>>> Result: (nil "te peer\nMAIL FROM:<my@address.edu> SIZE=374")
>>> Result: (nil "te peer\nMAIL FROM:<my@adress.edu> SIZE=374")
>>
>> Is that the value of the `result' variable?
>
> I believe it is.  am not too much of an expert...
>

interestingly, when using gmail for smtp (which works fine for me),
result gets the very different value

 (250 "250 2.1.0 OK somerandomlettershere.19")

is this helpful?

bug#9312: 24.0.50; smtpmail

[Lars Magne Ingebrigtsen]

emacs user <user.emacs@gmail.com> writes:

>>>> Result: (nil "te peer\nMAIL FROM:<my@address.edu> SIZE=374")
>>>> Result: (nil "te peer\nMAIL FROM:<my@adress.edu> SIZE=374")
>>>
>>> Is that the value of the `result' variable?
>>
>> I believe it is.  am not too much of an expert...
>>
>
> interestingly, when using gmail for smtp (which works fine for me),
> result gets the very different value
>
>  (250 "250 2.1.0 OK somerandomlettershere.19")

Interesting.  I wonder whether there's a race condition in
`smtpmail-read-response', because the thing you're getting back from
that function is just wrong.

Could you do the following?  After you've gotten the (nil "te peer")
thing back, could you switch to the "*trace of SMTP session ..."
buffer and post the entire contents of that buffer?

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

bug#9312: 24.0.50; smtpmail

[Lars Magne Ingebrigtsen]

I've now fixed a bug in smtpmail that may or may not be the cause of the
problems you're seeing.  Could you give it a try?  And if it still
doesn't work, post the "*trace ..." buffer.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

bug#9312: 24.0.50; smtpmail

[emacs user]

On Thu, Sep 15, 2011 at 2:07 AM, Lars Magne Ingebrigtsen <larsi@gnus.org> wrote:
> I've now fixed a bug in smtpmail that may or may not be the cause of the
> problems you're seeing.  Could you give it a try?  And if it still
> doesn't work, post the "*trace ..." buffer.

220 HUB00.xx.edu Microsoft ESMTP MAIL Service ready at Sat, 17 Sep
2011 11:07:52 -0400
250-HUB00.xx.edu Hello [xx.yy.39.xx]
250-SIZE 20971520
250-PIPELINING
250-DSN
250-ENHANCEDSTATUSCODES
250-STARTTLS
250-AUTH GSSAPI NTLM
250-8BITMIME
250-BINARYMIME
250 CHUNKING
220 2.0.0 SMTP server ready
250-HUB00.xx.edu Hello [xx.yy.39.xx]
250-SIZE 20971520
250-PIPELINING
250-DSN
250-ENHANCEDSTATUSCODES
250-AUTH GSSAPI NTLM LOGIN
250-8BITMIME
250-BINARYMIME
250 CHUNKING
MAIL FROM:<xx@yy.zz.edu> SIZE=375
530 5.7.1 Client was not authenticated

Process smtpmail connection broken by remote peer
QUIT
QUIT

bug#9312: 24.0.50; smtpmail

[Lars Magne Ingebrigtsen]

emacs user <user.emacs@gmail.com> writes:

> MAIL FROM:<xx@yy.zz.edu> SIZE=375
> 530 5.7.1 Client was not authenticated
>
> Process smtpmail connection broken by remote peer
> QUIT
> QUIT

Aha!  I now think I know what the problem is, and I may have fixed it in
the current bzr Emacs.  Could you try?

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

bug#9312: 24.0.50; smtpmail

[emacs user]

On Sun, Sep 18, 2011 at 10:37 AM, Lars Magne Ingebrigtsen wrote:
> emacs user <user.emacs@gmail.com> writes:
>
>> MAIL FROM:<xx@yy.zz.edu> SIZE=375
>> 530 5.7.1 Client was not authenticated
>>
>> Process smtpmail connection broken by remote peer
>> QUIT
>> QUIT
>
> Aha!  I now think I know what the problem is, and I may have fixed it in
> the current bzr Emacs.  Could you try?
>

indeed works!  brilliant of you, thanks for persisting!  cheers, E

bug#9312: 24.0.50; smtpmail

[Lars Magne Ingebrigtsen]

emacs user <user.emacs@gmail.com> writes:

>> Aha!  I now think I know what the problem is, and I may have fixed it in
>> the current bzr Emacs.  Could you try?
>
> indeed works!

Great!

Thanks for debugging this.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/